-- Logs begin at Wed 2022-06-08 01:08:32 UTC, end at Wed 2022-06-08 02:26:38 UTC. -- Jun 08 01:50:17.699054 dv-cinder-845069-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 08 01:50:17.708236 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 8 01:50:17 2022] *** Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: nodename: dv-cinder-845069-1-iscsi Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: machine: x86_64 Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: clock source: unix Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: pcre jit disabled Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: detected number of CPU cores: 4 Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: current working directory: / Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: detected binary path: /usr/bin/uwsgi-core Jun 08 01:50:17.710248 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: your processes number limit is 47680 Jun 08 01:50:17.710819 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: your memory page size is 4096 bytes Jun 08 01:50:17.710819 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: detected max file descriptor number: 2048 Jun 08 01:50:17.710819 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: lock engine: pthread robust mutexes Jun 08 01:50:17.710819 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: thunder lock: enabled Jun 08 01:50:17.710819 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 08 01:50:17.710819 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 08 01:50:17.755231 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: Python main interpreter initialized at 0x5619815aa9c0 Jun 08 01:50:17.755231 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: python threads support enabled Jun 08 01:50:17.755231 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: your server socket listen backlog is limited to 100 connections Jun 08 01:50:17.755231 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: your mercy for graceful operations on workers is 90 seconds Jun 08 01:50:17.755559 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: mapped 403077 bytes (393 KB) for 2 cores Jun 08 01:50:17.755651 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: *** Operational MODE: preforking *** Jun 08 01:50:17.755927 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: *** uWSGI is running in multiple interpreter mode *** Jun 08 01:50:17.755927 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: spawned uWSGI master process (pid: 80415) Jun 08 01:50:17.756115 dv-cinder-845069-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 08 01:50:17.756948 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: spawned uWSGI worker 1 (pid: 80425, cores: 1) Jun 08 01:50:17.757040 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: spawned uWSGI worker 2 (pid: 80426, cores: 1) Jun 08 01:50:17.760431 dv-cinder-845069-1-iscsi devstack@keystone.service[80415]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 08 01:50:23.493011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 08 01:50:23.493011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: from cryptography.utils import int_from_bytes Jun 08 01:50:23.493799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 08 01:50:23.493799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: from cryptography.utils import int_from_bytes Jun 08 01:50:23.494113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 08 01:50:23.494113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: from cryptography.utils import int_from_bytes Jun 08 01:50:23.494565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 08 01:50:23.494565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: from cryptography.utils import int_from_bytes Jun 08 01:50:23.543086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 08 01:50:23.543284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 08 01:50:23.543587 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] command line args: [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 08 01:50:23.543778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 08 01:50:23.543884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 08 01:50:23.543971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 08 01:50:23.544039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ================================================================================ {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 08 01:50:23.544168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] command line args: [] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 08 01:50:23.544299 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] admin_token = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.544457 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 08 01:50:23.544548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] config_dir = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.544637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ================================================================================ {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 08 01:50:23.544716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.544870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] config_source = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.544948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] admin_token = **** {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.545046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] debug = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.545133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] config_dir = [] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.545247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.545509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.545509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] config_source = [] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.545624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.545687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] debug = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.545761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.545822 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546301 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546542 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_config_append = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] list_limit = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.546994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_dir = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.547088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] log_config_append = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.547188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_file = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.547280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.547384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.547479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] log_dir = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.547578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.547655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] log_file = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.547817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.547898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.548055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.548136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.548258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.548335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.548474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.548547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.548663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.548788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.548863 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.548983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.549065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.549142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.549294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.549375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.549522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.549646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.549750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.549837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.549993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.550109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.550233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.550318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.550497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.550590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.550743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.550974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.551055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.551138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] publish_errors = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.551366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.551485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.551592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.551682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] publish_errors = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.551755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.551868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.551995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.552073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.552224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.552381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.552496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] standard_threads = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.552570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.552695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.552772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.552918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.552998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] standard_threads = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553119 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] use_journal = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553407 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] use_json = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] use_stderr = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553735 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] use_syslog = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] use_journal = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553897 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] use_json = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.553975 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.554075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] use_stderr = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.554195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.554273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] use_syslog = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.554428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.554518 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 08 01:50:23.554678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.554756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.554915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.554992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.555156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.555242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.555388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.555467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.555627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.555706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.555856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.556011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.556086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.556156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.556324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.556421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.556590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.556680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.556820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.556905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.557060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.557138 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.557289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.557375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.557528 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.557607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.557765 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.557893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.558000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.558111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.558245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.558325 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.558481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.558558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.558717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.558805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.558945 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.559536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.559751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.559830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.560089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.560225 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.560352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.560647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.560737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.560903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.560987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.561154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.561234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.561383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.561459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.561611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.561688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.561843 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.561922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.562083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.562167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.562325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.562400 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.562553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.562627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.562763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.562840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.563004 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.563090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.563247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.563330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.563480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.563558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.563716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.563804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.563940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.564018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.564171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.564249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.564418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.564511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.564645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.564723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.564870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.564947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.565094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.565183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.565322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.565398 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.565553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.565629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.565786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.565880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.566013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.566090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.566384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.566470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.566530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.566598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.566687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.566756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.566916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.566983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.567150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.567289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.567399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.567559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.567642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.567771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.567855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.567972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.568087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.568162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.568278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.568352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.568483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.568614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.568691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.568806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.568929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.569008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.569159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.569250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.569388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.569475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.569613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.569744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.569869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.569995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.570092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.570179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.570312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.570395 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.570543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.570620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.570774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.570877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.571032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.571113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.571265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.571369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.571506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.571624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.571746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.571833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.571978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.572059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.572208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.572297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.572454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.572537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.572693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.572772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.572933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.573025 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.573166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.573252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.573401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.573480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.573630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.573706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.573848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.573928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.574075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.574154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.574304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.574391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.574533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.574612 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.575369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.575461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.575642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.575733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.575891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.575991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.576136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.576224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.576376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.576479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.576645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.576724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.576885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.576964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.577123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.577214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.577360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.577441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.577584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.577680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.577814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.577917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.578075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.578163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.578243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.578315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.578477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.578555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.578706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.578786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.578938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.579016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.579167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.579244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.579410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.579490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.579661 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.579739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.579896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.579974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.580120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.external = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.580200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.580366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.580500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] auth.external = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.580614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.580693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.580894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.580983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.581102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.password = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.581183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.581326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] auth.token = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.581413 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] auth.password = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.581557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.581645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] auth.token = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.581785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.581871 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.582022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.582101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.582245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.582331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.582486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.582566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.582725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.582811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.582951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.583038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.583179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.583258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.583412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] credential.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.583491 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.583651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.583737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.583891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.583972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.584122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.584201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.584359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.584481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.584609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.584689 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.584851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.584930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.585099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.585185 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.585330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.585410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.585578 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.585656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.585829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.585916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.586097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.586176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.586342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.586420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.586586 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.586671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.586834 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.586914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.587063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.587142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.587296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.587384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] federation.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.587537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.587616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.587775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.587853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.588000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.588157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.588233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.588303 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.588484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.588567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.588722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.588813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.588953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.589031 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.589200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.589279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.589429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.589516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.589671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.589764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.589904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.590035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.590143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.590248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.590418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.590558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.590835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.591035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.592153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.592449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.592720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.592799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.592995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.593075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.593250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.593333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.593510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.593591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.593770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.593849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.594006 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.594088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.594236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.594316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.594471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.594550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.594717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.594799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.594966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.595045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.595215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.595294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.595466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.595545 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.595715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.595801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.595967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.596045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.596209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.596288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.596468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.596553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.596703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.596783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.596940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.597022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.597171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.597257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.597406 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.597486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.597643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.597723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.597878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.597956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.598119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.598197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.598344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.598424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.598588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.598706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.598832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.598912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.599065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.599146 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.599303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.599382 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.599544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.599623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.599773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.599854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.600008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.600087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.600233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.600312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.600489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.600573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.600726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.600816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.600964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.601043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.601198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.601279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.601434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.601515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.601673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.601752 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.601916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.601996 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.602144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.602222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.602370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.602448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.602622 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.602745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.602846 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.602925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.603080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.603159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.603313 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.603400 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.603544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.603624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.603773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.603852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.604004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.604084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.604245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.604325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.604494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.604575 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.604738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.604821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.604980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.605060 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.605221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.605309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.605454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.605534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.605686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.605765 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.605943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.606037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.606267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.606347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.606419 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.606490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.606707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.606795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.606948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.607027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.607189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.607267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.607431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.607519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.607675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.607755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.607916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.607996 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.608145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.608232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.608380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.608493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.608637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.608716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.608872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.608964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.609093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.609175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.609326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.609405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.609562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.609649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.609802 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.609880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.610038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.610117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.610274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.610361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.610501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.610579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.610742 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.610821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.610973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.611067 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.611208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.611321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.611431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.611510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.611648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.611736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.611886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.611966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.612113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.612200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.612344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.612522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.612590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.612648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.612834 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.612913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] resource.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.613076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.613168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.613308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.613388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.613546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.613624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.613789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.613875 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.614020 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.614105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.614263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.614341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.614497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.614587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.614732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.614825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.614961 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] role.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.615041 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.615195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] role.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.615282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] role.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.615421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.615506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] role.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.615660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.615745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.615899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.615987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.616141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.616221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.616383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.616484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.616644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.616736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.616885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.616964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.617126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.617205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.617368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.617449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.617600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.617679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.617837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.617922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.618077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.618157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.618317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.618397 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.618558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.618639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.618801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.618956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.619190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.619274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.619432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.619512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.619675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.619755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.619917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.619996 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.620159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.620239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.620473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.620560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.620655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.620733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.620901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.620980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.621138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.621219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.621364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.621443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.622110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.622224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.622334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.622464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.622580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.622788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.622868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.623048 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.623137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.623278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.623357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.623508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.623587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.623734 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.623814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.623962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.624042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.624194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.624273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.624455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.624540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] token.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.624689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.624769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.624936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.625015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.625176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.625258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.625406 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.625484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.625646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.625726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.625884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.625965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.626126 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.626204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.626355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.626434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.626587 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.626668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.626816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.626895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.627054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.627131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.627290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.627370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.627513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.627591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.627749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.627828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.627987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.628067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.628228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.628307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.connection = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.628530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.628624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.628774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.628855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.connection = **** {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.629008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.629101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.629242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.629323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.629470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.629550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.629703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.629791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.629935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.630015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.630165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.630245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.630405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.630493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.630637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.630717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.630876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.630964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.631105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.631194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.631348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.631427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.631578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.631657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.631818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.631975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.632052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.632124 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.632267 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.632347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.632517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.632613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.632688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 08 01:50:23.632760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.632982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 08 01:50:23.633157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 08 01:50:23.662288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.662515 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.662656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.662741 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.662891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.663114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.671187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.671470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.699066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.699290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.700694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.700694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.752820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.753169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.801454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.801685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.802227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.802467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.867522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.867745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.867832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.867965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.868070 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.868180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.868264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.868414 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.868499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.868632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.868718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.868872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.892192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.892304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.892501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.892588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.892730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.892813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.892953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.893048 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.893169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.893253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.893383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.893465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.893595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.893687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.904685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.904953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.917703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.917931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.918148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.918334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.918918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.919091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.919250 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.919334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.919470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.919543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.919686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.919809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.920040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.920262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 08 01:50:23.937414 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.937623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 08 01:50:23.937957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.938158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 08 01:50:23.938291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.938425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.938508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 08 01:50:23.938680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.938831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.938923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.939036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 08 01:50:23.939177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.939313 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.939576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.939835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.940092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.950208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.950657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.951099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.951229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.951499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.951626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.951753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.952038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.959971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:23.960346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:23.963237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:23.963512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.963651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:23.963776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.963910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.964040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.964194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.964322 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.964511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.964640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.964799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.964927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.965051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.965186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.965326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.965418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.965604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.965745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.965914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.966110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.966213 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.966406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.992783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.993270 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.994370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:23.994644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:23.994882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:23.995161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.000607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.000882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.001157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.001256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.001486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.001768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.008419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.008656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.009071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.009356 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.013162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.013444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.013720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.013898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.013998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.014188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.014272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.014472 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.014563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.014743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.014828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.015017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.015100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.015291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.015561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.015940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.033525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.033793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.034828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.035115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.038617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.039951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.042358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.042653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.044260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.044575 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.048160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.049608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.051737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.053232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.055349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.055640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.055919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.056198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.056509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.056790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.056877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.057153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.057439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.057723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.058011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.058302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.076588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.076870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.077139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.077396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.078274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.078562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.078832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.079096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.085273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.087017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.088383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.088695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.090256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.091029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.092550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.092832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.093107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.093378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.093644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.094121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.094402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.094684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.094975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.095264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.107099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.107377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.107652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.107927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.108209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.108513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.108687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.108792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.108973 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.109259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.109545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.109831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.110108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.110394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.126569 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.128309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.129819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.131581 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.133251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.134838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.135085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.136488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.136726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.136811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.137033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.138979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.139260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.139540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.144842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.146434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.146913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.148556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.149784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.150068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.150346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.150617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.151918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.152213 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.152527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.153364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.160220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.160528 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.160801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.162722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.163011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.163293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.169102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.169375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.169654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.169919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.170181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.170462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.170729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.170987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.171256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.171337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 08 01:50:24.171517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.171600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.171778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.171862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.172041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.172133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.172307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.172412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.172589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.172672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.172927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.173204 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.173487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.173773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.174043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.174315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.174594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.174854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 08 01:50:24.211024 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5619815aa9c0 pid: 80425 (default app) Jun 08 01:50:24.213931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5619815aa9c0 pid: 80426 (default app) Jun 08 01:50:24.214381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d50a49-d081-4bb8-811a-0cff63209bec None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:24.214668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d50a49-d081-4bb8-811a-0cff63209bec None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:24.214942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d50a49-d081-4bb8-811a-0cff63209bec None None] PATH_INFO: `/v3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:24.219317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Wed Jun 8 01:50:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:32.235040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcda902-fb96-4ad8-a06e-9ce5e73f491d None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:32.235572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcda902-fb96-4ad8-a06e-9ce5e73f491d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:32.235572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcda902-fb96-4ad8-a06e-9ce5e73f491d None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:32.239824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:32.244915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:32.245348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:32.245466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:32.313817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: INFO dbcounter [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] Registered counter for database keystone Jun 08 01:50:32.334537 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG oslo_db.sqlalchemy.engines [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80425) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 08 01:50:32.340862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writer thread running {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 08 01:50:32.620492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG passlib.handlers.bcrypt [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80425) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 08 01:50:32.620912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG passlib.handlers.bcrypt [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80425) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 08 01:50:32.676632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:32.678218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:32.800587 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 557 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 08 01:50:32.973731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcda902-fb96-4ad8-a06e-9ce5e73f491d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:33.043607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: INFO dbcounter [None req-4fcda902-fb96-4ad8-a06e-9ce5e73f491d None None] Registered counter for database keystone Jun 08 01:50:33.062931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG oslo_db.sqlalchemy.engines [None req-4fcda902-fb96-4ad8-a06e-9ce5e73f491d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80426) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 08 01:50:33.069107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writer thread running {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 08 01:50:33.295054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcda902-fb96-4ad8-a06e-9ce5e73f491d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:33.299487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9580ae-59ec-4728-917e-806b93e4327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:33.300156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9580ae-59ec-4728-917e-806b93e4327e admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:33.300471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9580ae-59ec-4728-917e-806b93e4327e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:33.300755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9580ae-59ec-4728-917e-806b93e4327e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:33.308968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9580ae-59ec-4728-917e-806b93e4327e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:33.360410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3a9580ae-59ec-4728-917e-806b93e4327e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 08 01:50:33.360767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9580ae-59ec-4728-917e-806b93e4327e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:33.370110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Jun 8 01:50:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 01:50:33.376260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:33.390930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ed470-2265-47e9-8b94-679f88df9d18 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:33.395375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79596516-aa72-4391-a620-d49eed0a2ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:33.396548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596516-aa72-4391-a620-d49eed0a2ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:33.396835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596516-aa72-4391-a620-d49eed0a2ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:33.397110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596516-aa72-4391-a620-d49eed0a2ebb admin admin] PATH_INFO: `/v3/projects/d34d18aa450b41d2ac1001da2d9a5476` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:33.398611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79596516-aa72-4391-a620-d49eed0a2ebb admin admin] RBAC: Authorizing `identity:get_project(project_id=d34d18aa450b41d2ac1001da2d9a5476)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:33.466226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: WARNING keystone.common.rbac_enforcer.enforcer [None req-79596516-aa72-4391-a620-d49eed0a2ebb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 08 01:50:33.466568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79596516-aa72-4391-a620-d49eed0a2ebb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:33.472538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Jun 8 01:50:33 2022] GET /identity/v3/projects/d34d18aa450b41d2ac1001da2d9a5476 => generated 344 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:35.805986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee852100-dbfe-45df-a84e-d6496b8a0cbc None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:35.806550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee852100-dbfe-45df-a84e-d6496b8a0cbc None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:35.806550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee852100-dbfe-45df-a84e-d6496b8a0cbc None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:35.810841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:35.816316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120470-af99-474d-a0ae-2a907391edfd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:35.816714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120470-af99-474d-a0ae-2a907391edfd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:35.816904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120470-af99-474d-a0ae-2a907391edfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:35.835835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-bd120470-af99-474d-a0ae-2a907391edfd None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:35.837386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-bd120470-af99-474d-a0ae-2a907391edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:35.846197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 08 01:50:35.868476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee852100-dbfe-45df-a84e-d6496b8a0cbc None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:35.880699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee852100-dbfe-45df-a84e-d6496b8a0cbc None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:35.884862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095376b-f81c-4c7b-a0e8-b14f0b572d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:35.885484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095376b-f81c-4c7b-a0e8-b14f0b572d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:35.885768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095376b-f81c-4c7b-a0e8-b14f0b572d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:35.886043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095376b-f81c-4c7b-a0e8-b14f0b572d98 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:35.894175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095376b-f81c-4c7b-a0e8-b14f0b572d98 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:35.895929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095376b-f81c-4c7b-a0e8-b14f0b572d98 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:35.905278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Jun 8 01:50:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 01:50:35.910646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120470-af99-474d-a0ae-2a907391edfd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:35.920584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120470-af99-474d-a0ae-2a907391edfd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:35.924958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e0d20-aed3-4094-be78-8b545eaf17f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:35.925818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e0d20-aed3-4094-be78-8b545eaf17f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:35.926101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e0d20-aed3-4094-be78-8b545eaf17f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:35.926372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e0d20-aed3-4094-be78-8b545eaf17f4 admin admin] PATH_INFO: `/v3/users/ded448ce0e8547839095b6fee4b26e38` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:35.927747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e0d20-aed3-4094-be78-8b545eaf17f4 admin admin] RBAC: Authorizing `identity:get_user(user_id=ded448ce0e8547839095b6fee4b26e38)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:35.929586 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e0d20-aed3-4094-be78-8b545eaf17f4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:35.931135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Wed Jun 8 01:50:35 2022] GET /identity/v3/users/ded448ce0e8547839095b6fee4b26e38 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:50:38.248975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377d3af-81ca-4b5c-959f-1bfd9db8d1b6 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:38.249475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377d3af-81ca-4b5c-959f-1bfd9db8d1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:38.249565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377d3af-81ca-4b5c-959f-1bfd9db8d1b6 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:38.250886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:50:38.252350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bb711-9942-4360-a238-5edd3cc04230 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:38.252785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bb711-9942-4360-a238-5edd3cc04230 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:38.252941 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bb711-9942-4360-a238-5edd3cc04230 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:38.254280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:50:38.255832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6209f-a955-48ba-9fe7-3e7e2b4e7485 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:38.256123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6209f-a955-48ba-9fe7-3e7e2b4e7485 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:38.256457 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6209f-a955-48ba-9fe7-3e7e2b4e7485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:38.259227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa0c11-eedf-4f9d-a085-05b920039120 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:38.259514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa0c11-eedf-4f9d-a085-05b920039120 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:38.259792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa0c11-eedf-4f9d-a085-05b920039120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:38.276440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fdfa0c11-eedf-4f9d-a085-05b920039120 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:38.278233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fdfa0c11-eedf-4f9d-a085-05b920039120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:38.297790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG passlib.handlers.bcrypt [None req-eda6209f-a955-48ba-9fe7-3e7e2b4e7485 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80426) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 08 01:50:38.298218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG passlib.handlers.bcrypt [None req-eda6209f-a955-48ba-9fe7-3e7e2b4e7485 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80426) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 08 01:50:38.345159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-eda6209f-a955-48ba-9fe7-3e7e2b4e7485 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:38.346733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-eda6209f-a955-48ba-9fe7-3e7e2b4e7485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:38.358146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 7/12] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:50:38.380214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa0c11-eedf-4f9d-a085-05b920039120 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:38.404654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:38.447668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 6/13] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:38.471993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6209f-a955-48ba-9fe7-3e7e2b4e7485 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:38.483108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa0c11-eedf-4f9d-a085-05b920039120 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:38.516246 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf2ff5-502c-40b5-95f2-d49e5074a337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:38.516861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf2ff5-502c-40b5-95f2-d49e5074a337 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:38.517144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf2ff5-502c-40b5-95f2-d49e5074a337 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:38.517419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf2ff5-502c-40b5-95f2-d49e5074a337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:38.572823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6209f-a955-48ba-9fe7-3e7e2b4e7485 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:38.575678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:38.584579 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf2ff5-502c-40b5-95f2-d49e5074a337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:38.586082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf2ff5-502c-40b5-95f2-d49e5074a337 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:38.610060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:38.612359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98669e-5ce4-4f7d-98e4-fab8b83044eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:38.613422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98669e-5ce4-4f7d-98e4-fab8b83044eb None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:38.613713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98669e-5ce4-4f7d-98e4-fab8b83044eb None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:38.613994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98669e-5ce4-4f7d-98e4-fab8b83044eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:38.625858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98669e-5ce4-4f7d-98e4-fab8b83044eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-6d98669e-5ce4-4f7d-98e4-fab8b83044eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:38.637231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:38.638233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:50:38.643815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:50:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:50:38.644483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98669e-5ce4-4f7d-98e4-fab8b83044eb None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:38.687277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 8/15] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:50:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 08 01:50:38.692732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf2ff5-502c-40b5-95f2-d49e5074a337 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:38.743112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98669e-5ce4-4f7d-98e4-fab8b83044eb None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:38.778878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c065f7-d2bf-4e6a-a56f-f3ce17acc513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:38.779462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c065f7-d2bf-4e6a-a56f-f3ce17acc513 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:38.779746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c065f7-d2bf-4e6a-a56f-f3ce17acc513 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:38.780019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c065f7-d2bf-4e6a-a56f-f3ce17acc513 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:38.780800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c065f7-d2bf-4e6a-a56f-f3ce17acc513 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:38.781106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c065f7-d2bf-4e6a-a56f-f3ce17acc513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:38.782550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c065f7-d2bf-4e6a-a56f-f3ce17acc513 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:38.789134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf2ff5-502c-40b5-95f2-d49e5074a337 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:38.794936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 8/16] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:50:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:38.800509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c065f7-d2bf-4e6a-a56f-f3ce17acc513 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:38.814021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:38.826205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66c8a0-100c-4cf6-ae0a-099ffa78d3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:38.827010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66c8a0-100c-4cf6-ae0a-099ffa78d3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:38.827297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66c8a0-100c-4cf6-ae0a-099ffa78d3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:38.827573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66c8a0-100c-4cf6-ae0a-099ffa78d3f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:38.827719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:38.838240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66c8a0-100c-4cf6-ae0a-099ffa78d3f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:38.840290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66c8a0-100c-4cf6-ae0a-099ffa78d3f3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2d66c8a0-100c-4cf6-ae0a-099ffa78d3f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:50:38.861543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:38.862367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:38.863145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:50:38.863145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:50:38.863145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:50:38.863145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:50:38.863145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:50:38.868436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66c8a0-100c-4cf6-ae0a-099ffa78d3f3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:38.903083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c065f7-d2bf-4e6a-a56f-f3ce17acc513 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:38.937652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38464f9d-deb7-4ba3-a6f7-11452ef89f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:38.938533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38464f9d-deb7-4ba3-a6f7-11452ef89f0a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:38.938817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38464f9d-deb7-4ba3-a6f7-11452ef89f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:38.939120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38464f9d-deb7-4ba3-a6f7-11452ef89f0a None admin] PATH_INFO: `/v3/users/ded448ce0e8547839095b6fee4b26e38` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:38.940720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38464f9d-deb7-4ba3-a6f7-11452ef89f0a None admin] RBAC: Authorizing `identity:get_user(user_id=ded448ce0e8547839095b6fee4b26e38)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:38.942203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38464f9d-deb7-4ba3-a6f7-11452ef89f0a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:38.943824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:50:38 2022] GET /identity/v3/users/ded448ce0e8547839095b6fee4b26e38 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:50:38.949717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38464f9d-deb7-4ba3-a6f7-11452ef89f0a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:38.970950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66c8a0-100c-4cf6-ae0a-099ffa78d3f3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:39.004085 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d4ce5-b640-43b1-a2ae-701da9882e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:39.004779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d4ce5-b640-43b1-a2ae-701da9882e0e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:39.005068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d4ce5-b640-43b1-a2ae-701da9882e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:39.005350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d4ce5-b640-43b1-a2ae-701da9882e0e None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:39.006067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d4ce5-b640-43b1-a2ae-701da9882e0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:39.006534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d4ce5-b640-43b1-a2ae-701da9882e0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:39.007940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d4ce5-b640-43b1-a2ae-701da9882e0e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:39.025504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:50:39.043920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:39.051814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38464f9d-deb7-4ba3-a6f7-11452ef89f0a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:39.088037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9e17a-0a73-48fb-90e4-d2a2f905111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:39.088890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9e17a-0a73-48fb-90e4-d2a2f905111c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:39.089180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9e17a-0a73-48fb-90e4-d2a2f905111c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:39.089591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9e17a-0a73-48fb-90e4-d2a2f905111c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:39.091722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9e17a-0a73-48fb-90e4-d2a2f905111c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:39.093282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9e17a-0a73-48fb-90e4-d2a2f905111c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:39.094989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:50:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:50:39.101016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d4ce5-b640-43b1-a2ae-701da9882e0e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:39.138291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:39.208591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d4ce5-b640-43b1-a2ae-701da9882e0e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:39.243050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8045b1-d786-4299-a669-8b4d4c49a140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:39.243716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8045b1-d786-4299-a669-8b4d4c49a140 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:39.244000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8045b1-d786-4299-a669-8b4d4c49a140 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:39.244274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8045b1-d786-4299-a669-8b4d4c49a140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:39.245130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8045b1-d786-4299-a669-8b4d4c49a140 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.domain.id=default, user.id=ded448ce0e8547839095b6fee4b26e38) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:39.245441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8045b1-d786-4299-a669-8b4d4c49a140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:39.247468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8045b1-d786-4299-a669-8b4d4c49a140 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:39.254284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Jun 8 01:50:39 2022] GET /identity/v3/role_assignments?user.id=ded448ce0e8547839095b6fee4b26e38&scope.domain.id=default&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:41.700584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5946a7-2bce-46dd-9e21-07c3e942cac0 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:41.700584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5946a7-2bce-46dd-9e21-07c3e942cac0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:41.700584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5946a7-2bce-46dd-9e21-07c3e942cac0 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:41.700584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:41.706166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea0f71-a36f-4b40-bc6a-a977489d7a9c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:41.706977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea0f71-a36f-4b40-bc6a-a977489d7a9c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:41.707481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea0f71-a36f-4b40-bc6a-a977489d7a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:41.729401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b6ea0f71-a36f-4b40-bc6a-a977489d7a9c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:41.731055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b6ea0f71-a36f-4b40-bc6a-a977489d7a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:41.823147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:41.869935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5946a7-2bce-46dd-9e21-07c3e942cac0 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:42.022785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5946a7-2bce-46dd-9e21-07c3e942cac0 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:42.073625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96515892-eb5e-4354-aed2-071e29d2665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:42.074820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96515892-eb5e-4354-aed2-071e29d2665c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.075301 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96515892-eb5e-4354-aed2-071e29d2665c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.075797 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96515892-eb5e-4354-aed2-071e29d2665c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.102886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96515892-eb5e-4354-aed2-071e29d2665c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:42.104852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96515892-eb5e-4354-aed2-071e29d2665c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-96515892-eb5e-4354-aed2-071e29d2665c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:50:42.130759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:42.131630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:42.132507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:50:42.132507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:50:42.132507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:50:42.132507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:50:42.134522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 12/24] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:50:42.141533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea0f71-a36f-4b40-bc6a-a977489d7a9c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:42.155975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e20a7-0c71-40c0-ac5e-011aa6ec106e None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.156629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e20a7-0c71-40c0-ac5e-011aa6ec106e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.157128 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e20a7-0c71-40c0-ac5e-011aa6ec106e None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.158760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 13/25] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:50:42.169781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb52fc7-3870-4cf3-8498-d44c378b775a None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.170399 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb52fc7-3870-4cf3-8498-d44c378b775a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.170875 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb52fc7-3870-4cf3-8498-d44c378b775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.182846 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:42.215404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-6cb52fc7-3870-4cf3-8498-d44c378b775a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:42.217335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6cb52fc7-3870-4cf3-8498-d44c378b775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:42.233310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:42.326272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 14/26] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:42.328120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf3edd-9842-4785-bd4e-696b610c9b6a None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.330204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea0f71-a36f-4b40-bc6a-a977489d7a9c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:42.332045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf3edd-9842-4785-bd4e-696b610c9b6a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.332568 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf3edd-9842-4785-bd4e-696b610c9b6a None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.335502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 15/27] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:42.346383 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635a64-2404-4ddc-9a48-91e0af0f1b47 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.346941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635a64-2404-4ddc-9a48-91e0af0f1b47 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.347437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635a64-2404-4ddc-9a48-91e0af0f1b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.377528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-eb635a64-2404-4ddc-9a48-91e0af0f1b47 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:42.378728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-eb635a64-2404-4ddc-9a48-91e0af0f1b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:42.397747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc472b8d-b12c-45bb-af02-b17c175fdb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:42.398419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc472b8d-b12c-45bb-af02-b17c175fdb47 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.398702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc472b8d-b12c-45bb-af02-b17c175fdb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.398974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc472b8d-b12c-45bb-af02-b17c175fdb47 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.399699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc472b8d-b12c-45bb-af02-b17c175fdb47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:42.400003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc472b8d-b12c-45bb-af02-b17c175fdb47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:42.401431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc472b8d-b12c-45bb-af02-b17c175fdb47 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:42.432667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 13/28] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:42.433856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc472b8d-b12c-45bb-af02-b17c175fdb47 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:42.526160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 16/29] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:42.527538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635a64-2404-4ddc-9a48-91e0af0f1b47 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:42.571529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:42.587172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc472b8d-b12c-45bb-af02-b17c175fdb47 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:42.623627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:42.630990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a71af-6175-4595-8e8c-e303e9cab5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:42.631778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a71af-6175-4595-8e8c-e303e9cab5f3 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.632193 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a71af-6175-4595-8e8c-e303e9cab5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.632577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a71af-6175-4595-8e8c-e303e9cab5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.633526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a71af-6175-4595-8e8c-e303e9cab5f3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:42.634992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a71af-6175-4595-8e8c-e303e9cab5f3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:42.656387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635a64-2404-4ddc-9a48-91e0af0f1b47 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:42.697389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa6a36-5875-458f-87b9-6630b78fd7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:42.698147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa6a36-5875-458f-87b9-6630b78fd7d5 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.698499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa6a36-5875-458f-87b9-6630b78fd7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.699354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa6a36-5875-458f-87b9-6630b78fd7d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.700145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa6a36-5875-458f-87b9-6630b78fd7d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:42.701846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa6a36-5875-458f-87b9-6630b78fd7d5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:42.777706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 14/30] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Jun 8 01:50:42 2022] POST /identity/v3/roles => generated 225 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:50:42.779412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782a52-a50b-4e5f-ac9e-00fd31a42d11 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.779710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782a52-a50b-4e5f-ac9e-00fd31a42d11 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.780281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7782a52-a50b-4e5f-ac9e-00fd31a42d11 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.786146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 15/31] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:42.787327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7782a52-a50b-4e5f-ac9e-00fd31a42d11 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:42.845118 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 17/32] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:50:42 2022] POST /identity/v3/projects => generated 301 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:50:42.850551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9226d09-6753-443a-80b9-ab676224ead1 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.850845 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9226d09-6753-443a-80b9-ab676224ead1 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.851128 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9226d09-6753-443a-80b9-ab676224ead1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.882435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-f9226d09-6753-443a-80b9-ab676224ead1 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:42.884046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f9226d09-6753-443a-80b9-ab676224ead1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:42.919041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7782a52-a50b-4e5f-ac9e-00fd31a42d11 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:42.952153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50934bfb-f617-4924-8a5b-fa780dce1f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:42.953120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934bfb-f617-4924-8a5b-fa780dce1f42 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.953599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934bfb-f617-4924-8a5b-fa780dce1f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.954052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934bfb-f617-4924-8a5b-fa780dce1f42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.973726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50934bfb-f617-4924-8a5b-fa780dce1f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-50934bfb-f617-4924-8a5b-fa780dce1f42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:42.983003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:42.983923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:50:42.984707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:50:42.985286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 16/33] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:50:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:50:42.988060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1349625-5437-457a-9a07-9c5b2d8805ac None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.988060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1349625-5437-457a-9a07-9c5b2d8805ac None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.988060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1349625-5437-457a-9a07-9c5b2d8805ac None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:42.988060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 17/34] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:42.994690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984c556-b890-4246-a58c-1319d2ae5265 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:42.994968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984c556-b890-4246-a58c-1319d2ae5265 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:42.995257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984c556-b890-4246-a58c-1319d2ae5265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.018292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 18/35] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:43.019241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9226d09-6753-443a-80b9-ab676224ead1 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:43.035498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5984c556-b890-4246-a58c-1319d2ae5265 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:43.040351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5984c556-b890-4246-a58c-1319d2ae5265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:43.043411 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:43.135109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 18/36] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:43.136177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984c556-b890-4246-a58c-1319d2ae5265 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:43.150505 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:43.231040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9226d09-6753-443a-80b9-ab676224ead1 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:43.267328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984c556-b890-4246-a58c-1319d2ae5265 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:43.276263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351272b4-2629-403b-951e-7309db0ed5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:43.277486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351272b4-2629-403b-951e-7309db0ed5fe None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:43.277814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351272b4-2629-403b-951e-7309db0ed5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:43.278097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351272b4-2629-403b-951e-7309db0ed5fe None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.278860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351272b4-2629-403b-951e-7309db0ed5fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:43.279171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351272b4-2629-403b-951e-7309db0ed5fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:43.280743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351272b4-2629-403b-951e-7309db0ed5fe None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:43.290195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 19/37] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:50:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:43.294203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351272b4-2629-403b-951e-7309db0ed5fe None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:43.305513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97830e-cba5-470d-af9c-0ba32f334a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:43.306198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97830e-cba5-470d-af9c-0ba32f334a8f None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:43.306488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97830e-cba5-470d-af9c-0ba32f334a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:43.306771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97830e-cba5-470d-af9c-0ba32f334a8f None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.307787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97830e-cba5-470d-af9c-0ba32f334a8f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:43.310165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97830e-cba5-470d-af9c-0ba32f334a8f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:43.321129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:43.336427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 19/38] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Jun 8 01:50:43 2022] POST /identity/v3/roles => generated 219 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:50:43.337822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97830e-cba5-470d-af9c-0ba32f334a8f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:43.416592 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351272b4-2629-403b-951e-7309db0ed5fe None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:43.437731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:43.466965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e564219-ca4a-4edf-b441-dae790d8e184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:43.467789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e564219-ca4a-4edf-b441-dae790d8e184 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:43.468074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e564219-ca4a-4edf-b441-dae790d8e184 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:43.468350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e564219-ca4a-4edf-b441-dae790d8e184 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.469688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e564219-ca4a-4edf-b441-dae790d8e184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:43.471216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e564219-ca4a-4edf-b441-dae790d8e184 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:43.472986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 20/39] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:50:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:43.479852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e564219-ca4a-4edf-b441-dae790d8e184 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:43.499480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97830e-cba5-470d-af9c-0ba32f334a8f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:43.537545 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35f38b-385b-4437-8691-a076c9ea89a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:43.538440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35f38b-385b-4437-8691-a076c9ea89a2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:43.538728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35f38b-385b-4437-8691-a076c9ea89a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:43.539006 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35f38b-385b-4437-8691-a076c9ea89a2 None admin] PATH_INFO: `/v3/users/ded448ce0e8547839095b6fee4b26e38` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.540292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35f38b-385b-4437-8691-a076c9ea89a2 None admin] RBAC: Authorizing `identity:get_user(user_id=ded448ce0e8547839095b6fee4b26e38)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:43.541819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35f38b-385b-4437-8691-a076c9ea89a2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:43.544001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 20/40] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:50:43 2022] GET /identity/v3/users/ded448ce0e8547839095b6fee4b26e38 => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:43.545337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8bd57-8eb1-4f23-9103-b83fffd6bde7 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:43.545723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8bd57-8eb1-4f23-9103-b83fffd6bde7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:43.546006 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8bd57-8eb1-4f23-9103-b83fffd6bde7 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.551158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 21/41] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:43.551421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8bd57-8eb1-4f23-9103-b83fffd6bde7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:43.604147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e564219-ca4a-4edf-b441-dae790d8e184 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:43.629915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:43.642255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4212a-8127-4d5b-a526-1147834371d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:43.642924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4212a-8127-4d5b-a526-1147834371d6 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:43.643216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4212a-8127-4d5b-a526-1147834371d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:43.643498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4212a-8127-4d5b-a526-1147834371d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.644218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4212a-8127-4d5b-a526-1147834371d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:43.645758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4212a-8127-4d5b-a526-1147834371d6 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:43.669877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8bd57-8eb1-4f23-9103-b83fffd6bde7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:43.672135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 21/42] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:50:43 2022] POST /identity/v3/projects => generated 312 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:50:43.673248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e87838-5558-4924-9561-ebff764baf1c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:43.673545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e87838-5558-4924-9561-ebff764baf1c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:43.673827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e87838-5558-4924-9561-ebff764baf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.691599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-40e87838-5558-4924-9561-ebff764baf1c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:43.692861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-40e87838-5558-4924-9561-ebff764baf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:43.716068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c408d5-c14b-46f7-ac80-195b869d5314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:43.716950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c408d5-c14b-46f7-ac80-195b869d5314 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:43.717238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c408d5-c14b-46f7-ac80-195b869d5314 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:43.717521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c408d5-c14b-46f7-ac80-195b869d5314 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.718840 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c408d5-c14b-46f7-ac80-195b869d5314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:43.720346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c408d5-c14b-46f7-ac80-195b869d5314 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:43.722052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 22/43] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:50:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:43.727692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c408d5-c14b-46f7-ac80-195b869d5314 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:43.729472 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:43.775772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 22/44] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:43.804283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e87838-5558-4924-9561-ebff764baf1c None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:43.855777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c408d5-c14b-46f7-ac80-195b869d5314 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:43.897908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f17a52-aceb-43c6-8651-844b34264079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:43.899402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f17a52-aceb-43c6-8651-844b34264079 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:43.899873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f17a52-aceb-43c6-8651-844b34264079 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:43.900342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f17a52-aceb-43c6-8651-844b34264079 None admin] PATH_INFO: `/v3/domains/default/users/ded448ce0e8547839095b6fee4b26e38/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.903061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f17a52-aceb-43c6-8651-844b34264079 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ded448ce0e8547839095b6fee4b26e38, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:43.904984 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f17a52-aceb-43c6-8651-844b34264079 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:43.907045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:43.925247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e87838-5558-4924-9561-ebff764baf1c None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:43.925753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 23/45] 10.222.0.20 () {58 vars in 1361 bytes} [Wed Jun 8 01:50:43 2022] PUT /identity/v3/domains/default/users/ded448ce0e8547839095b6fee4b26e38/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 01:50:43.967839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d9643a-92f3-4961-99eb-3024e7f40462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:43.968605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9643a-92f3-4961-99eb-3024e7f40462 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:43.968898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9643a-92f3-4961-99eb-3024e7f40462 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:43.969181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9643a-92f3-4961-99eb-3024e7f40462 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:43.969964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d9643a-92f3-4961-99eb-3024e7f40462 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:43.971418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d9643a-92f3-4961-99eb-3024e7f40462 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:43.987331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 23/46] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Jun 8 01:50:43 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:50:45.968012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf7c04-e429-4e03-8e85-55979ccaa421 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:45.968525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf7c04-e429-4e03-8e85-55979ccaa421 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:45.968633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf7c04-e429-4e03-8e85-55979ccaa421 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:45.969974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 24/47] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:45.975261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ca756-8340-4b48-a7a9-1f8ece07de4a None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:45.975627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ca756-8340-4b48-a7a9-1f8ece07de4a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:45.975833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ca756-8340-4b48-a7a9-1f8ece07de4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:45.993252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7e9ca756-8340-4b48-a7a9-1f8ece07de4a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:45.994434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7e9ca756-8340-4b48-a7a9-1f8ece07de4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:46.016990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:46.062779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 24/48] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:50:46.085964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf7c04-e429-4e03-8e85-55979ccaa421 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:46.173344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ae67a-3c14-4bdc-bbb2-b6fa64b6ee47 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:46.173563 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ae67a-3c14-4bdc-bbb2-b6fa64b6ee47 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:46.173855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ae67a-3c14-4bdc-bbb2-b6fa64b6ee47 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:46.175234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 25/49] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:50:46.180233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66e99d-58f9-421d-832a-6c1db2138b6e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:46.180566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66e99d-58f9-421d-832a-6c1db2138b6e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:46.180858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66e99d-58f9-421d-832a-6c1db2138b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:46.190343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf7c04-e429-4e03-8e85-55979ccaa421 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:46.197248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-9f66e99d-58f9-421d-832a-6c1db2138b6e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:46.198449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-9f66e99d-58f9-421d-832a-6c1db2138b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:46.224027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f8ff9-8fbd-48ba-8ebb-48f5dad5f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:46.224866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f8ff9-8fbd-48ba-8ebb-48f5dad5f390 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:46.225153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f8ff9-8fbd-48ba-8ebb-48f5dad5f390 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:46.225432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f8ff9-8fbd-48ba-8ebb-48f5dad5f390 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:46.226603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f8ff9-8fbd-48ba-8ebb-48f5dad5f390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:46.228070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f8ff9-8fbd-48ba-8ebb-48f5dad5f390 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:46.229714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 25/50] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:50:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:46.235715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f8ff9-8fbd-48ba-8ebb-48f5dad5f390 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:46.266480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 26/51] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:50:46.290414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66e99d-58f9-421d-832a-6c1db2138b6e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:46.296756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:46.336573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f8ff9-8fbd-48ba-8ebb-48f5dad5f390 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:46.370208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44be1ce-b9be-4dcc-88c2-78517f35a0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:46.370922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44be1ce-b9be-4dcc-88c2-78517f35a0be None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:46.371209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44be1ce-b9be-4dcc-88c2-78517f35a0be None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:46.371492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44be1ce-b9be-4dcc-88c2-78517f35a0be None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:46.372196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44be1ce-b9be-4dcc-88c2-78517f35a0be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:46.373709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44be1ce-b9be-4dcc-88c2-78517f35a0be None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:46.377679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:46.390930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66e99d-58f9-421d-832a-6c1db2138b6e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:46.400852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 26/52] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Jun 8 01:50:46 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:50:46.428499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b1a95-e63f-4a64-bfab-b2f5b5333760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:46.429268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b1a95-e63f-4a64-bfab-b2f5b5333760 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:46.429553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b1a95-e63f-4a64-bfab-b2f5b5333760 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:46.429834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b1a95-e63f-4a64-bfab-b2f5b5333760 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:46.442702 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b1a95-e63f-4a64-bfab-b2f5b5333760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-8b6b1a95-e63f-4a64-bfab-b2f5b5333760 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:46.450683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:46.451617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:50:46.452436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 27/53] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:50:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:50:46.459304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44be1ce-b9be-4dcc-88c2-78517f35a0be None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:46.582980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44be1ce-b9be-4dcc-88c2-78517f35a0be None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:46.597621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:46.624256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2fe1e-5dc3-43ed-8e55-afef45eca0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:46.625213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2fe1e-5dc3-43ed-8e55-afef45eca0fe None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:46.625334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2fe1e-5dc3-43ed-8e55-afef45eca0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:46.625656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2fe1e-5dc3-43ed-8e55-afef45eca0fe None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:46.626456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2fe1e-5dc3-43ed-8e55-afef45eca0fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:46.626810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2fe1e-5dc3-43ed-8e55-afef45eca0fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:46.628382 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2fe1e-5dc3-43ed-8e55-afef45eca0fe None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:46.640738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 27/54] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:46.646360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b1a95-e63f-4a64-bfab-b2f5b5333760 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:46.745435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b1a95-e63f-4a64-bfab-b2f5b5333760 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:46.778763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee007e4c-d02f-4819-a0ec-a58da8171be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:46.779639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee007e4c-d02f-4819-a0ec-a58da8171be5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:46.779929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee007e4c-d02f-4819-a0ec-a58da8171be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:46.780212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee007e4c-d02f-4819-a0ec-a58da8171be5 None admin] PATH_INFO: `/v3/users/ded448ce0e8547839095b6fee4b26e38` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:46.781440 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee007e4c-d02f-4819-a0ec-a58da8171be5 None admin] RBAC: Authorizing `identity:get_user(user_id=ded448ce0e8547839095b6fee4b26e38)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:46.782918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee007e4c-d02f-4819-a0ec-a58da8171be5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:46.784583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 28/55] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:50:46 2022] GET /identity/v3/users/ded448ce0e8547839095b6fee4b26e38 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:46.790213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2fe1e-5dc3-43ed-8e55-afef45eca0fe None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:46.893392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2fe1e-5dc3-43ed-8e55-afef45eca0fe None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:46.928967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b27159-765a-4657-bae5-210d3b8efa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:46.929802 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b27159-765a-4657-bae5-210d3b8efa9d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:46.930089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b27159-765a-4657-bae5-210d3b8efa9d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:46.930367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b27159-765a-4657-bae5-210d3b8efa9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:46.931546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b27159-765a-4657-bae5-210d3b8efa9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:46.933146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b27159-765a-4657-bae5-210d3b8efa9d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:46.934924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 28/56] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:50:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:50:46.940573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee007e4c-d02f-4819-a0ec-a58da8171be5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:46.959148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:47.041977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee007e4c-d02f-4819-a0ec-a58da8171be5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:47.075821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cff2b-6d1d-4504-8ce9-fa1d4520ec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:47.076514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cff2b-6d1d-4504-8ce9-fa1d4520ec01 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:47.076807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cff2b-6d1d-4504-8ce9-fa1d4520ec01 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:47.077090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cff2b-6d1d-4504-8ce9-fa1d4520ec01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:47.077926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cff2b-6d1d-4504-8ce9-fa1d4520ec01 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.domain.id=default, user.id=ded448ce0e8547839095b6fee4b26e38) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:47.078244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cff2b-6d1d-4504-8ce9-fa1d4520ec01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:47.079708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cff2b-6d1d-4504-8ce9-fa1d4520ec01 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:47.088402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 29/57] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Jun 8 01:50:46 2022] GET /identity/v3/role_assignments?user.id=ded448ce0e8547839095b6fee4b26e38&scope.domain.id=default&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:48.634692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bf02b-3421-42a3-a174-289d960869c1 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:48.635235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bf02b-3421-42a3-a174-289d960869c1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:48.635235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bf02b-3421-42a3-a174-289d960869c1 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:48.637146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 29/58] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:48.642469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2110a-0598-4855-adb0-089590bc5069 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:48.642851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2110a-0598-4855-adb0-089590bc5069 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:48.643046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2110a-0598-4855-adb0-089590bc5069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:48.660149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-dde2110a-0598-4855-adb0-089590bc5069 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:48.661364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-dde2110a-0598-4855-adb0-089590bc5069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:48.726732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 30/59] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:48.750692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bf02b-3421-42a3-a174-289d960869c1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:48.797976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:48.851693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bf02b-3421-42a3-a174-289d960869c1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:48.889042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d148dd3-d226-46b8-ae00-6c48eab52066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:48.889756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d148dd3-d226-46b8-ae00-6c48eab52066 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:48.890040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d148dd3-d226-46b8-ae00-6c48eab52066 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:48.890316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d148dd3-d226-46b8-ae00-6c48eab52066 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:48.891480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d148dd3-d226-46b8-ae00-6c48eab52066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:48.892984 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d148dd3-d226-46b8-ae00-6c48eab52066 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:48.894586 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 30/60] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:50:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:48.900167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2110a-0598-4855-adb0-089590bc5069 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:48.913281 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:49.000980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2110a-0598-4855-adb0-089590bc5069 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:49.034113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8861cb3-c305-482e-8a1c-e4717db3b595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:49.034778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8861cb3-c305-482e-8a1c-e4717db3b595 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:49.035064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8861cb3-c305-482e-8a1c-e4717db3b595 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:49.035343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8861cb3-c305-482e-8a1c-e4717db3b595 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:49.036085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8861cb3-c305-482e-8a1c-e4717db3b595 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:49.037856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8861cb3-c305-482e-8a1c-e4717db3b595 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:49.062577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 31/61] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:50:48 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:50:52.552225 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de77ea-1c87-4665-98c0-b150c2046c3a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:52.553251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de77ea-1c87-4665-98c0-b150c2046c3a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:52.553737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de77ea-1c87-4665-98c0-b150c2046c3a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:52.555561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 31/62] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:52.561455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36375-656b-4e9a-bdf6-e17f4b8e4645 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:52.562305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36375-656b-4e9a-bdf6-e17f4b8e4645 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:52.562784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a36375-656b-4e9a-bdf6-e17f4b8e4645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:52.593411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b9a36375-656b-4e9a-bdf6-e17f4b8e4645 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:52.594954 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b9a36375-656b-4e9a-bdf6-e17f4b8e4645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:52.675726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:52.692226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 32/63] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:50:52.730115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de77ea-1c87-4665-98c0-b150c2046c3a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:52.754074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6da97d-c768-41f3-b89f-385f9e32e329 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:52.754695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6da97d-c768-41f3-b89f-385f9e32e329 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:52.755190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6da97d-c768-41f3-b89f-385f9e32e329 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:52.756823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 33/64] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:52.762628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c9073-5795-428a-b64e-199cd2cc22d2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:52.763240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c9073-5795-428a-b64e-199cd2cc22d2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:52.763719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c9073-5795-428a-b64e-199cd2cc22d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:52.783551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-893c9073-5795-428a-b64e-199cd2cc22d2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:52.785241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-893c9073-5795-428a-b64e-199cd2cc22d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:52.884453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 34/65] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:52.886184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c346-1cfe-43c1-99e7-cf5d6a6e0289 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:52.886832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c346-1cfe-43c1-99e7-cf5d6a6e0289 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:52.886935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de77ea-1c87-4665-98c0-b150c2046c3a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:52.887540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c346-1cfe-43c1-99e7-cf5d6a6e0289 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:52.890278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 35/66] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:52.896156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af5744-0dce-4947-a57c-102ff70e1e6d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:52.897174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af5744-0dce-4947-a57c-102ff70e1e6d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:52.897684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af5744-0dce-4947-a57c-102ff70e1e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:52.901036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:52.922680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-18af5744-0dce-4947-a57c-102ff70e1e6d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:52.924087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-18af5744-0dce-4947-a57c-102ff70e1e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:52.926139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a9da9-e3ac-4b41-a23e-6c37c8a5bb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:52.926943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a9da9-e3ac-4b41-a23e-6c37c8a5bb8e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:52.927230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a9da9-e3ac-4b41-a23e-6c37c8a5bb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:52.927507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a9da9-e3ac-4b41-a23e-6c37c8a5bb8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:52.938974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a9da9-e3ac-4b41-a23e-6c37c8a5bb8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-908a9da9-e3ac-4b41-a23e-6c37c8a5bb8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:52.946620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:52.947470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 32/67] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:50:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:50:52.953244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46034c08-3375-4ddb-8567-f24b0f4f383c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:52.954392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46034c08-3375-4ddb-8567-f24b0f4f383c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:52.954392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46034c08-3375-4ddb-8567-f24b0f4f383c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:52.954392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 33/68] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:52.954392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46034c08-3375-4ddb-8567-f24b0f4f383c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.007436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 36/69] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:53.008284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af5744-0dce-4947-a57c-102ff70e1e6d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.068670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46034c08-3375-4ddb-8567-f24b0f4f383c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.070419 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:53.105677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69eaad-c0c9-49f6-bc5c-73ab660b357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.106493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69eaad-c0c9-49f6-bc5c-73ab660b357f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.106785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69eaad-c0c9-49f6-bc5c-73ab660b357f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.107198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69eaad-c0c9-49f6-bc5c-73ab660b357f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.118206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69eaad-c0c9-49f6-bc5c-73ab660b357f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:53.122379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af5744-0dce-4947-a57c-102ff70e1e6d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5d69eaad-c0c9-49f6-bc5c-73ab660b357f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:53.126122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:53.127140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:50:53.128099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 34/70] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:50:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:50:53.128913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f42865-8d2e-4f52-9c27-a3f8c700ee44 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.128913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f42865-8d2e-4f52-9c27-a3f8c700ee44 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.129089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f42865-8d2e-4f52-9c27-a3f8c700ee44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.146561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-67f42865-8d2e-4f52-9c27-a3f8c700ee44 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:53.147695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-67f42865-8d2e-4f52-9c27-a3f8c700ee44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:53.162413 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f09cfe-f952-4973-a8c6-24db816351a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.163079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f09cfe-f952-4973-a8c6-24db816351a7 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.163366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f09cfe-f952-4973-a8c6-24db816351a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.163645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f09cfe-f952-4973-a8c6-24db816351a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.164436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f09cfe-f952-4973-a8c6-24db816351a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:53.164728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f09cfe-f952-4973-a8c6-24db816351a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:53.166123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f09cfe-f952-4973-a8c6-24db816351a7 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:53.174019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:53.174317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 37/71] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:53.175475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f09cfe-f952-4973-a8c6-24db816351a7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.219489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 35/72] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:53.220872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388f0a-6642-45e9-8c59-91864c72fbb9 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.221167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388f0a-6642-45e9-8c59-91864c72fbb9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.221447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388f0a-6642-45e9-8c59-91864c72fbb9 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.222717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 36/73] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:53.223421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b388f0a-6642-45e9-8c59-91864c72fbb9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.300508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f09cfe-f952-4973-a8c6-24db816351a7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.317807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:53.332853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b388f0a-6642-45e9-8c59-91864c72fbb9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.335537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1af11-75a7-4454-b1a8-5c46351ff2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.336349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1af11-75a7-4454-b1a8-5c46351ff2ac None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.336661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1af11-75a7-4454-b1a8-5c46351ff2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.336942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1af11-75a7-4454-b1a8-5c46351ff2ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.347887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1af11-75a7-4454-b1a8-5c46351ff2ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-9ae1af11-75a7-4454-b1a8-5c46351ff2ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:53.356913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:53.358593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 38/74] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:50:53.360352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1af11-75a7-4454-b1a8-5c46351ff2ac None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.375655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc61770-ffde-4f11-9961-162cb33e0470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.376330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc61770-ffde-4f11-9961-162cb33e0470 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.376641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc61770-ffde-4f11-9961-162cb33e0470 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.376917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc61770-ffde-4f11-9961-162cb33e0470 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.377665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc61770-ffde-4f11-9961-162cb33e0470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:53.377967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc61770-ffde-4f11-9961-162cb33e0470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:53.379364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc61770-ffde-4f11-9961-162cb33e0470 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:53.387462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 37/75] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:53.388668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297834e-280d-46e0-b2fc-0fa27e7dad80 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.388956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297834e-280d-46e0-b2fc-0fa27e7dad80 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.389228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297834e-280d-46e0-b2fc-0fa27e7dad80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.406100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7297834e-280d-46e0-b2fc-0fa27e7dad80 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:53.407770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7297834e-280d-46e0-b2fc-0fa27e7dad80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:53.425414 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:53.467345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1af11-75a7-4454-b1a8-5c46351ff2ac None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.476670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 38/76] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:53.480947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297834e-280d-46e0-b2fc-0fa27e7dad80 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.502333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbba3f-58e5-4a29-828c-8075a06876c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.503214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbba3f-58e5-4a29-828c-8075a06876c5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.503499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbba3f-58e5-4a29-828c-8075a06876c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.503778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbba3f-58e5-4a29-828c-8075a06876c5 None admin] PATH_INFO: `/v3/users/9dd12833b3ae4101a15287c432ec51b3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.520211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbba3f-58e5-4a29-828c-8075a06876c5 None admin] RBAC: Authorizing `identity:get_user(user_id=9dd12833b3ae4101a15287c432ec51b3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:53.521711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbba3f-58e5-4a29-828c-8075a06876c5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:53.523369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 39/77] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/users/9dd12833b3ae4101a15287c432ec51b3 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:53.524134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbba3f-58e5-4a29-828c-8075a06876c5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.559878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:53.584912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297834e-280d-46e0-b2fc-0fa27e7dad80 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.618206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ff061-aac2-4cc3-947e-676663008e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.619003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ff061-aac2-4cc3-947e-676663008e35 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.619292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ff061-aac2-4cc3-947e-676663008e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.619569 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ff061-aac2-4cc3-947e-676663008e35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.623998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbba3f-58e5-4a29-828c-8075a06876c5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.630779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ff061-aac2-4cc3-947e-676663008e35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-043ff061-aac2-4cc3-947e-676663008e35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:53.638597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:53.639449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:50:53.640220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 39/78] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:50:53.640881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ff061-aac2-4cc3-947e-676663008e35 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.656469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:53.663318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71448f1-ef4c-4137-bbf0-a7a71389e489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.663988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71448f1-ef4c-4137-bbf0-a7a71389e489 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.664274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71448f1-ef4c-4137-bbf0-a7a71389e489 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.664582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71448f1-ef4c-4137-bbf0-a7a71389e489 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.665325 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71448f1-ef4c-4137-bbf0-a7a71389e489 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:53.665634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71448f1-ef4c-4137-bbf0-a7a71389e489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:53.667005 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71448f1-ef4c-4137-bbf0-a7a71389e489 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:53.675860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 40/79] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:53.677379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71448f1-ef4c-4137-bbf0-a7a71389e489 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.742475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ff061-aac2-4cc3-947e-676663008e35 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.772262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:53.776185 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71448f1-ef4c-4137-bbf0-a7a71389e489 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.776298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6508a38-75de-423d-b580-3a05fcf407d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.777672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6508a38-75de-423d-b580-3a05fcf407d1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.778115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6508a38-75de-423d-b580-3a05fcf407d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.778214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6508a38-75de-423d-b580-3a05fcf407d1 None admin] PATH_INFO: `/v3/users/ded448ce0e8547839095b6fee4b26e38` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.779448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6508a38-75de-423d-b580-3a05fcf407d1 None admin] RBAC: Authorizing `identity:get_user(user_id=ded448ce0e8547839095b6fee4b26e38)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:53.781336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6508a38-75de-423d-b580-3a05fcf407d1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:53.782601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 40/80] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/users/ded448ce0e8547839095b6fee4b26e38 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:53.783370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6508a38-75de-423d-b580-3a05fcf407d1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.813796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65632e14-8ba5-41e0-8809-3c03505f392b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.814695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65632e14-8ba5-41e0-8809-3c03505f392b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.814987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65632e14-8ba5-41e0-8809-3c03505f392b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.815272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65632e14-8ba5-41e0-8809-3c03505f392b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.816464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65632e14-8ba5-41e0-8809-3c03505f392b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:53.817918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65632e14-8ba5-41e0-8809-3c03505f392b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:53.819528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 41/81] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:53.821247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65632e14-8ba5-41e0-8809-3c03505f392b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.865905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:53.883745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6508a38-75de-423d-b580-3a05fcf407d1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.919303 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af63eb9-4f59-41d1-a90e-de1777173c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.919965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af63eb9-4f59-41d1-a90e-de1777173c6d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.920254 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af63eb9-4f59-41d1-a90e-de1777173c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.920565 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af63eb9-4f59-41d1-a90e-de1777173c6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.923030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65632e14-8ba5-41e0-8809-3c03505f392b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:53.957148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082281c-27f1-4c6d-85a0-443cfff8ef70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:53.957848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082281c-27f1-4c6d-85a0-443cfff8ef70 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:53.958163 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082281c-27f1-4c6d-85a0-443cfff8ef70 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:53.958442 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082281c-27f1-4c6d-85a0-443cfff8ef70 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:53.959193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082281c-27f1-4c6d-85a0-443cfff8ef70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:53.959504 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082281c-27f1-4c6d-85a0-443cfff8ef70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:53.960939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082281c-27f1-4c6d-85a0-443cfff8ef70 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:53.969089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 42/82] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:53.969877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082281c-27f1-4c6d-85a0-443cfff8ef70 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:53.993172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af63eb9-4f59-41d1-a90e-de1777173c6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:53.994671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af63eb9-4f59-41d1-a90e-de1777173c6d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:53.997956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:54.068333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082281c-27f1-4c6d-85a0-443cfff8ef70 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:54.077188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:54.096687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 41/83] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:50:54.097623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af63eb9-4f59-41d1-a90e-de1777173c6d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:54.103893 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c9f18-e701-4fb4-bf82-89c5d9994ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:54.105526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c9f18-e701-4fb4-bf82-89c5d9994ecd None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:54.105892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c9f18-e701-4fb4-bf82-89c5d9994ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:54.106177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c9f18-e701-4fb4-bf82-89c5d9994ecd None admin] PATH_INFO: `/v3/users/9dd12833b3ae4101a15287c432ec51b3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:54.107380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c9f18-e701-4fb4-bf82-89c5d9994ecd None admin] RBAC: Authorizing `identity:get_user(user_id=9dd12833b3ae4101a15287c432ec51b3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:54.108896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c9f18-e701-4fb4-bf82-89c5d9994ecd None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:54.110528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 43/84] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:50:53 2022] GET /identity/v3/users/9dd12833b3ae4101a15287c432ec51b3 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:54.111276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c9f18-e701-4fb4-bf82-89c5d9994ecd None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:54.198935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af63eb9-4f59-41d1-a90e-de1777173c6d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:54.205512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:54.211452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c9f18-e701-4fb4-bf82-89c5d9994ecd None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:54.233532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d832c-3faf-47c4-82ee-7d34381cc45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:54.234195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d832c-3faf-47c4-82ee-7d34381cc45e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:54.234485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d832c-3faf-47c4-82ee-7d34381cc45e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:54.234775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d832c-3faf-47c4-82ee-7d34381cc45e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:54.245982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc4acd-06c6-413b-8dc2-b2c35bb85745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:54.246650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc4acd-06c6-413b-8dc2-b2c35bb85745 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:54.246939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc4acd-06c6-413b-8dc2-b2c35bb85745 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:54.247221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc4acd-06c6-413b-8dc2-b2c35bb85745 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:54.247934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc4acd-06c6-413b-8dc2-b2c35bb85745 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:54.249439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc4acd-06c6-413b-8dc2-b2c35bb85745 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:54.268278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 44/85] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:50:54 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:50:54.273437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc4acd-06c6-413b-8dc2-b2c35bb85745 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:54.288598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:54.315511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d832c-3faf-47c4-82ee-7d34381cc45e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:54.317025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d832c-3faf-47c4-82ee-7d34381cc45e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:54.396613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc4acd-06c6-413b-8dc2-b2c35bb85745 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:54.434068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 42/86] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:50:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:50:54.434953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d832c-3faf-47c4-82ee-7d34381cc45e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:54.445328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bb01a-8b57-467d-a5f4-0e96a23ba87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:54.446309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bb01a-8b57-467d-a5f4-0e96a23ba87d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:54.446549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bb01a-8b57-467d-a5f4-0e96a23ba87d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:54.446828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bb01a-8b57-467d-a5f4-0e96a23ba87d None admin] PATH_INFO: `/v3/users/9dd12833b3ae4101a15287c432ec51b3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:54.448115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bb01a-8b57-467d-a5f4-0e96a23ba87d None admin] RBAC: Authorizing `identity:get_user(user_id=9dd12833b3ae4101a15287c432ec51b3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:54.450743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bb01a-8b57-467d-a5f4-0e96a23ba87d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:54.452603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 45/87] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:50:54 2022] GET /identity/v3/users/9dd12833b3ae4101a15287c432ec51b3 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:54.453386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bb01a-8b57-467d-a5f4-0e96a23ba87d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:54.479473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:54.542219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:54.572149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d832c-3faf-47c4-82ee-7d34381cc45e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:54.585752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bb01a-8b57-467d-a5f4-0e96a23ba87d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:54.611880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff407b90-b349-47bf-9a6b-d9e9e4eac651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:54.612843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff407b90-b349-47bf-9a6b-d9e9e4eac651 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:54.613185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff407b90-b349-47bf-9a6b-d9e9e4eac651 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:54.613411 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff407b90-b349-47bf-9a6b-d9e9e4eac651 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:54.614865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff407b90-b349-47bf-9a6b-d9e9e4eac651 None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:54.616373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff407b90-b349-47bf-9a6b-d9e9e4eac651 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:54.623738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 43/88] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:50:54 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:54.624932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff407b90-b349-47bf-9a6b-d9e9e4eac651 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:54.627975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a856988-a043-441d-8fcf-f503c3d2b964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:54.628677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a856988-a043-441d-8fcf-f503c3d2b964 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:54.628973 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a856988-a043-441d-8fcf-f503c3d2b964 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:54.629260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a856988-a043-441d-8fcf-f503c3d2b964 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:54.713569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a856988-a043-441d-8fcf-f503c3d2b964 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:54.715422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a856988-a043-441d-8fcf-f503c3d2b964 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:54.745078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:54.761727 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff407b90-b349-47bf-9a6b-d9e9e4eac651 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:54.802878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6fcb3-1e4f-4621-9362-5aa4580dcd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:54.804068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6fcb3-1e4f-4621-9362-5aa4580dcd27 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:54.804364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6fcb3-1e4f-4621-9362-5aa4580dcd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:54.804678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6fcb3-1e4f-4621-9362-5aa4580dcd27 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:54.806176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6fcb3-1e4f-4621-9362-5aa4580dcd27 None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:54.807687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6fcb3-1e4f-4621-9362-5aa4580dcd27 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:54.810147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 44/89] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:50:54 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:54.812514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6fcb3-1e4f-4621-9362-5aa4580dcd27 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:54.834951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:54.866872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 46/90] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:50:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:50:54.868040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a856988-a043-441d-8fcf-f503c3d2b964 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:54.930726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6fcb3-1e4f-4621-9362-5aa4580dcd27 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:54.981542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd5911-941d-479b-89ff-21ba33831ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:54.982516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd5911-941d-479b-89ff-21ba33831ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:54.982910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd5911-941d-479b-89ff-21ba33831ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:54.983290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd5911-941d-479b-89ff-21ba33831ba6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:54.993126 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a856988-a043-441d-8fcf-f503c3d2b964 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:55.015989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:55.035013 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543670bf-8f8c-4f02-b79d-d1b7a6a57247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:55.035867 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543670bf-8f8c-4f02-b79d-d1b7a6a57247 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:55.036316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543670bf-8f8c-4f02-b79d-d1b7a6a57247 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:55.037011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543670bf-8f8c-4f02-b79d-d1b7a6a57247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:55.037979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543670bf-8f8c-4f02-b79d-d1b7a6a57247 None admin] RBAC: Adding query filter params (role.id=61490cf1d7334421b84ff67552b79bfc, scope.project.id=faa8ebf890584746af20166b89a7e162, user.id=9dd12833b3ae4101a15287c432ec51b3) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:55.038433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543670bf-8f8c-4f02-b79d-d1b7a6a57247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:55.040160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543670bf-8f8c-4f02-b79d-d1b7a6a57247 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:55.046948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 47/91] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:50:54 2022] GET /identity/v3/role_assignments?user.id=9dd12833b3ae4101a15287c432ec51b3&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=61490cf1d7334421b84ff67552b79bfc => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:55.048176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543670bf-8f8c-4f02-b79d-d1b7a6a57247 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:55.102480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd5911-941d-479b-89ff-21ba33831ba6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:55.105432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd5911-941d-479b-89ff-21ba33831ba6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:55.147334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:55.252348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543670bf-8f8c-4f02-b79d-d1b7a6a57247 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:55.293657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 45/92] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:50:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:50:55.295011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd5911-941d-479b-89ff-21ba33831ba6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:55.315789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1adb72-d173-4e58-b1dd-ba11a6712010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:55.316510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1adb72-d173-4e58-b1dd-ba11a6712010 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:55.316802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1adb72-d173-4e58-b1dd-ba11a6712010 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:55.317082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1adb72-d173-4e58-b1dd-ba11a6712010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:55.317926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1adb72-d173-4e58-b1dd-ba11a6712010 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=faa8ebf890584746af20166b89a7e162, user.id=ded448ce0e8547839095b6fee4b26e38) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:55.318242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1adb72-d173-4e58-b1dd-ba11a6712010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:55.319709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1adb72-d173-4e58-b1dd-ba11a6712010 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:55.326016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 48/93] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:50:55 2022] GET /identity/v3/role_assignments?user.id=ded448ce0e8547839095b6fee4b26e38&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:55.326627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1adb72-d173-4e58-b1dd-ba11a6712010 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:55.385151 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:55.416153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd5911-941d-479b-89ff-21ba33831ba6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:55.436291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:55.456420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4eab04-44a4-46fa-b0de-1b145e4cb9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:55.457645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eab04-44a4-46fa-b0de-1b145e4cb9af None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:55.458123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eab04-44a4-46fa-b0de-1b145e4cb9af None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:55.458591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eab04-44a4-46fa-b0de-1b145e4cb9af None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:55.459157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1adb72-d173-4e58-b1dd-ba11a6712010 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:55.460291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4eab04-44a4-46fa-b0de-1b145e4cb9af None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:55.461989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4eab04-44a4-46fa-b0de-1b145e4cb9af None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:55.464600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 46/94] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:50:55 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:55.470737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4eab04-44a4-46fa-b0de-1b145e4cb9af None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:55.524091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632fe85-1bae-484d-b656-c7eb6a529bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:55.527652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632fe85-1bae-484d-b656-c7eb6a529bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:55.530184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632fe85-1bae-484d-b656-c7eb6a529bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:55.532016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632fe85-1bae-484d-b656-c7eb6a529bc5 None admin] PATH_INFO: `/v3/projects/2837e57df6a94a86a7e8d7c2ec65ee48` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:55.533732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632fe85-1bae-484d-b656-c7eb6a529bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=2837e57df6a94a86a7e8d7c2ec65ee48)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:55.536854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632fe85-1bae-484d-b656-c7eb6a529bc5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:55.538925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 49/95] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:50:55 2022] GET /identity/v3/projects/2837e57df6a94a86a7e8d7c2ec65ee48 => generated 312 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:55.545697 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632fe85-1bae-484d-b656-c7eb6a529bc5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:55.665186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4eab04-44a4-46fa-b0de-1b145e4cb9af None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:55.706589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c70a8-4783-4f0a-8e61-a236b48afb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:55.707284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c70a8-4783-4f0a-8e61-a236b48afb29 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:55.707573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c70a8-4783-4f0a-8e61-a236b48afb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:55.707853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c70a8-4783-4f0a-8e61-a236b48afb29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:55.709105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c70a8-4783-4f0a-8e61-a236b48afb29 None admin] RBAC: Adding query filter params (role.id=c45c9440d2b8447ab50b814c6be35bf1, scope.project.id=faa8ebf890584746af20166b89a7e162, user.id=9dd12833b3ae4101a15287c432ec51b3) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:55.709434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c70a8-4783-4f0a-8e61-a236b48afb29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:55.710908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c70a8-4783-4f0a-8e61-a236b48afb29 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:55.722764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 47/96] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:50:55 2022] GET /identity/v3/role_assignments?user.id=9dd12833b3ae4101a15287c432ec51b3&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=c45c9440d2b8447ab50b814c6be35bf1 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:50:55.775153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632fe85-1bae-484d-b656-c7eb6a529bc5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:55.803239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:55.829144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e53f0-c6b3-4f7b-86af-ac0b750f8074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:55.829829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e53f0-c6b3-4f7b-86af-ac0b750f8074 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:55.830116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e53f0-c6b3-4f7b-86af-ac0b750f8074 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:55.830398 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e53f0-c6b3-4f7b-86af-ac0b750f8074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:55.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e53f0-c6b3-4f7b-86af-ac0b750f8074 None admin] RBAC: Adding query filter params (role.id=61490cf1d7334421b84ff67552b79bfc, scope.project.id=2837e57df6a94a86a7e8d7c2ec65ee48, user.id=9dd12833b3ae4101a15287c432ec51b3) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:55.831580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e53f0-c6b3-4f7b-86af-ac0b750f8074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:55.833127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e53f0-c6b3-4f7b-86af-ac0b750f8074 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:55.840736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 50/97] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:50:55 2022] GET /identity/v3/role_assignments?user.id=9dd12833b3ae4101a15287c432ec51b3&scope.project.id=2837e57df6a94a86a7e8d7c2ec65ee48&role.id=61490cf1d7334421b84ff67552b79bfc => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:56.775619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ebde1-9bdc-440b-a0e3-8746df344cf6 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:56.776185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ebde1-9bdc-440b-a0e3-8746df344cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:56.776185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ebde1-9bdc-440b-a0e3-8746df344cf6 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:56.778135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 48/98] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:56.790397 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20562c1e-3acc-4c4c-8afd-9f380618ae9d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:56.790918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20562c1e-3acc-4c4c-8afd-9f380618ae9d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:56.791069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20562c1e-3acc-4c4c-8afd-9f380618ae9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:56.813508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-20562c1e-3acc-4c4c-8afd-9f380618ae9d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:56.814719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-20562c1e-3acc-4c4c-8afd-9f380618ae9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:56.897733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 51/99] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:50:56.942443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ebde1-9bdc-440b-a0e3-8746df344cf6 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:57.019648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:57.084701 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ebde1-9bdc-440b-a0e3-8746df344cf6 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:57.130283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb221b4-02b6-4cd4-be54-a44615af1a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:57.131101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb221b4-02b6-4cd4-be54-a44615af1a8a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:57.131387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb221b4-02b6-4cd4-be54-a44615af1a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:57.131664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb221b4-02b6-4cd4-be54-a44615af1a8a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:57.142991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb221b4-02b6-4cd4-be54-a44615af1a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-efb221b4-02b6-4cd4-be54-a44615af1a8a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:57.150698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:57.152310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:50:57.153686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 49/100] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:50:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:50:57.158288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20562c1e-3acc-4c4c-8afd-9f380618ae9d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:57.271238 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20562c1e-3acc-4c4c-8afd-9f380618ae9d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:57.283961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:57.311546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c2c1d-ec91-40e9-b996-9a2ff7d02152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:57.312231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c2c1d-ec91-40e9-b996-9a2ff7d02152 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:57.312556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c2c1d-ec91-40e9-b996-9a2ff7d02152 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:57.312839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c2c1d-ec91-40e9-b996-9a2ff7d02152 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:57.313595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c2c1d-ec91-40e9-b996-9a2ff7d02152 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:57.313905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c2c1d-ec91-40e9-b996-9a2ff7d02152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:57.315298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c2c1d-ec91-40e9-b996-9a2ff7d02152 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:57.324091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 52/101] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:50:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:57.329566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb221b4-02b6-4cd4-be54-a44615af1a8a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:57.341497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf24f6-34a6-46f9-8a75-ecd5a0269f98 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:57.341741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf24f6-34a6-46f9-8a75-ecd5a0269f98 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:57.341985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf24f6-34a6-46f9-8a75-ecd5a0269f98 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:57.345555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 53/102] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:57.349797 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adbc13-e072-41b3-8e68-0eb25bf925c2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:57.350311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adbc13-e072-41b3-8e68-0eb25bf925c2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:57.350816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adbc13-e072-41b3-8e68-0eb25bf925c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:57.388832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e5adbc13-e072-41b3-8e68-0eb25bf925c2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:57.394668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e5adbc13-e072-41b3-8e68-0eb25bf925c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:57.534697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb221b4-02b6-4cd4-be54-a44615af1a8a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:57.538930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 54/103] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:57.550801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:57.582881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5adbc13-e072-41b3-8e68-0eb25bf925c2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:57.604245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697ff97-5fa5-46e7-a35e-e6de085826a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:57.605510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697ff97-5fa5-46e7-a35e-e6de085826a1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:57.605812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697ff97-5fa5-46e7-a35e-e6de085826a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:57.606097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697ff97-5fa5-46e7-a35e-e6de085826a1 None admin] PATH_INFO: `/v3/users/eaa273ae467c455db58a3326415ce7ea` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:57.633994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697ff97-5fa5-46e7-a35e-e6de085826a1 None admin] RBAC: Authorizing `identity:get_user(user_id=eaa273ae467c455db58a3326415ce7ea)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:57.635500 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697ff97-5fa5-46e7-a35e-e6de085826a1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:57.638295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 50/104] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:50:57 2022] GET /identity/v3/users/eaa273ae467c455db58a3326415ce7ea => generated 289 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:57.644129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697ff97-5fa5-46e7-a35e-e6de085826a1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:57.777552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5adbc13-e072-41b3-8e68-0eb25bf925c2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:57.784426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:57.838493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260ce52-0760-4b66-86a6-d20914a4f00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:57.840226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260ce52-0760-4b66-86a6-d20914a4f00d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:57.840728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260ce52-0760-4b66-86a6-d20914a4f00d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:57.841209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260ce52-0760-4b66-86a6-d20914a4f00d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:57.845374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260ce52-0760-4b66-86a6-d20914a4f00d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:57.847158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260ce52-0760-4b66-86a6-d20914a4f00d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:57.853881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 55/105] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:50:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:50:57.856735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260ce52-0760-4b66-86a6-d20914a4f00d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:57.868565 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697ff97-5fa5-46e7-a35e-e6de085826a1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:57.924553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc2e50-1cba-4607-aeb8-2c55ee55826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:57.924553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2e50-1cba-4607-aeb8-2c55ee55826f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:57.924553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2e50-1cba-4607-aeb8-2c55ee55826f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:57.924553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc2e50-1cba-4607-aeb8-2c55ee55826f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:57.991205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:58.026754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc2e50-1cba-4607-aeb8-2c55ee55826f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:58.028919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dc2e50-1cba-4607-aeb8-2c55ee55826f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:58.114425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260ce52-0760-4b66-86a6-d20914a4f00d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:58.137044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 51/106] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 08 01:50:58.142572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc2e50-1cba-4607-aeb8-2c55ee55826f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:58.221797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:58.245645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01fac7-7342-45e0-8247-e26f1782318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:58.246840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01fac7-7342-45e0-8247-e26f1782318b None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:58.247322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01fac7-7342-45e0-8247-e26f1782318b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:58.247807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01fac7-7342-45e0-8247-e26f1782318b None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:58.248759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01fac7-7342-45e0-8247-e26f1782318b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:58.250452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01fac7-7342-45e0-8247-e26f1782318b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:58.256433 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc2e50-1cba-4607-aeb8-2c55ee55826f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:58.316570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3bd5e-d01e-4d51-adfa-329540eb1e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:58.317710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3bd5e-d01e-4d51-adfa-329540eb1e2b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:58.318063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3bd5e-d01e-4d51-adfa-329540eb1e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:58.318401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3bd5e-d01e-4d51-adfa-329540eb1e2b None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:58.325284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3bd5e-d01e-4d51-adfa-329540eb1e2b None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:58.326823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3bd5e-d01e-4d51-adfa-329540eb1e2b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:58.327546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 56/107] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:50:57 2022] POST /identity/v3/projects => generated 302 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:50:58.328861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 52/108] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:50:58 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:58.334557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01fac7-7342-45e0-8247-e26f1782318b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:58.384854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:58.505482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01fac7-7342-45e0-8247-e26f1782318b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:58.551515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32a377-ac39-4754-9d92-044985dcf8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:58.552200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32a377-ac39-4754-9d92-044985dcf8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:58.552519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32a377-ac39-4754-9d92-044985dcf8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:58.552805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32a377-ac39-4754-9d92-044985dcf8a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:58.553665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32a377-ac39-4754-9d92-044985dcf8a4 None admin] RBAC: Adding query filter params (role.id=e9245ca6f01f444dbbd7fe5e6121449c, scope.project.id=faa8ebf890584746af20166b89a7e162, user.id=eaa273ae467c455db58a3326415ce7ea) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:58.553790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736f665-c5b0-4b38-a2b3-abbf5194f5b7 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:58.553981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32a377-ac39-4754-9d92-044985dcf8a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:58.554076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736f665-c5b0-4b38-a2b3-abbf5194f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:58.554342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736f665-c5b0-4b38-a2b3-abbf5194f5b7 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:58.555476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32a377-ac39-4754-9d92-044985dcf8a4 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:58.555881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 53/109] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:58.561346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5ff12-f8d3-451c-ac38-64ab9e1e97b8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:58.561640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5ff12-f8d3-451c-ac38-64ab9e1e97b8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:58.561923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5ff12-f8d3-451c-ac38-64ab9e1e97b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:58.565252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 57/110] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:50:58 2022] GET /identity/v3/role_assignments?user.id=eaa273ae467c455db58a3326415ce7ea&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=e9245ca6f01f444dbbd7fe5e6121449c => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:58.593629 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ccd5ff12-f8d3-451c-ac38-64ab9e1e97b8 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:58.594882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ccd5ff12-f8d3-451c-ac38-64ab9e1e97b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:58.694133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 54/111] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:50:58.728249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32a377-ac39-4754-9d92-044985dcf8a4 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:58.855333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:58.874678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32a377-ac39-4754-9d92-044985dcf8a4 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:58.934820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0faa7-0247-4c42-8e51-45104aede165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:58.935653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0faa7-0247-4c42-8e51-45104aede165 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:58.935941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0faa7-0247-4c42-8e51-45104aede165 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:58.936215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0faa7-0247-4c42-8e51-45104aede165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:58.952120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0faa7-0247-4c42-8e51-45104aede165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-bfd0faa7-0247-4c42-8e51-45104aede165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:58.960020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:58.960976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:50:58.961740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 58/112] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:50:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:50:58.972091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5ff12-f8d3-451c-ac38-64ab9e1e97b8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:59.090333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:59.139116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5ff12-f8d3-451c-ac38-64ab9e1e97b8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:59.166033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3a698-f425-46e9-8e99-1747fd7d9bf9 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.166664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3a698-f425-46e9-8e99-1747fd7d9bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.167672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3a698-f425-46e9-8e99-1747fd7d9bf9 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.169409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 59/113] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:59.175038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2947b2-bc51-4090-b4d5-4109fab5902b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.175602 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2947b2-bc51-4090-b4d5-4109fab5902b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.176077 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2947b2-bc51-4090-b4d5-4109fab5902b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.187741 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf736c5-9a09-4b58-80a8-645d54808569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:59.188471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf736c5-9a09-4b58-80a8-645d54808569 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.188766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf736c5-9a09-4b58-80a8-645d54808569 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.189044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf736c5-9a09-4b58-80a8-645d54808569 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.190712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf736c5-9a09-4b58-80a8-645d54808569 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:59.190712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf736c5-9a09-4b58-80a8-645d54808569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:59.191674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf736c5-9a09-4b58-80a8-645d54808569 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:59.208300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 55/114] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:50:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:59.213936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3b2947b2-bc51-4090-b4d5-4109fab5902b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:59.214323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf736c5-9a09-4b58-80a8-645d54808569 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:59.215749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3b2947b2-bc51-4090-b4d5-4109fab5902b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:59.368343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf736c5-9a09-4b58-80a8-645d54808569 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:59.406385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 60/115] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:50:59.407825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a9d10-529d-4538-a18f-d8dddd72286c None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.408121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a9d10-529d-4538-a18f-d8dddd72286c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.408422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a9d10-529d-4538-a18f-d8dddd72286c None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.409944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 61/116] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:59.415488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3b77f-0e25-4e39-bd13-1d5d0872f19d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.415787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3b77f-0e25-4e39-bd13-1d5d0872f19d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.416094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3b77f-0e25-4e39-bd13-1d5d0872f19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.417872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:59.433483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75419da-eed8-42f5-8be7-64a808999b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:59.434871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75419da-eed8-42f5-8be7-64a808999b4c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.435161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75419da-eed8-42f5-8be7-64a808999b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.435493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75419da-eed8-42f5-8be7-64a808999b4c None admin] PATH_INFO: `/v3/users/ded448ce0e8547839095b6fee4b26e38` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.436824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75419da-eed8-42f5-8be7-64a808999b4c None admin] RBAC: Authorizing `identity:get_user(user_id=ded448ce0e8547839095b6fee4b26e38)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:59.439903 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75419da-eed8-42f5-8be7-64a808999b4c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:59.442083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 56/117] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:50:59 2022] GET /identity/v3/users/ded448ce0e8547839095b6fee4b26e38 => generated 247 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:50:59.444023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-91f3b77f-0e25-4e39-bd13-1d5d0872f19d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:59.445253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-91f3b77f-0e25-4e39-bd13-1d5d0872f19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:50:59.446656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75419da-eed8-42f5-8be7-64a808999b4c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:59.545303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 62/118] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:50:59.546178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3b77f-0e25-4e39-bd13-1d5d0872f19d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:59.554211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:59.596595 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75419da-eed8-42f5-8be7-64a808999b4c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:59.636897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a72479-97bd-429e-ade7-673280f1b99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:59.638057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a72479-97bd-429e-ade7-673280f1b99f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.638555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a72479-97bd-429e-ade7-673280f1b99f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.639036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a72479-97bd-429e-ade7-673280f1b99f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.653112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a72479-97bd-429e-ade7-673280f1b99f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-45a72479-97bd-429e-ade7-673280f1b99f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:59.664872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:59.665773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:50:59.666518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:50:59.669220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 57/119] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:50:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:50:59.674042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3b77f-0e25-4e39-bd13-1d5d0872f19d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:59.674183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a662b-d951-43ed-b5c2-f899b41b701e None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.674183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a662b-d951-43ed-b5c2-f899b41b701e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.674433 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a662b-d951-43ed-b5c2-f899b41b701e None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.675832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 58/120] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:50:59.677137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a662b-d951-43ed-b5c2-f899b41b701e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:59.719733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9da85-a515-4835-b079-ac1fff4b70ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:59.721169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9da85-a515-4835-b079-ac1fff4b70ac None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.721662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9da85-a515-4835-b079-ac1fff4b70ac None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.722913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9da85-a515-4835-b079-ac1fff4b70ac None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.724379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9da85-a515-4835-b079-ac1fff4b70ac None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:59.726092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9da85-a515-4835-b079-ac1fff4b70ac None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:59.728486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 63/121] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:50:59 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:59.730132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9da85-a515-4835-b079-ac1fff4b70ac None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:59.751567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:59.813474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a662b-d951-43ed-b5c2-f899b41b701e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:59.819418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:50:59.849947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9da85-a515-4835-b079-ac1fff4b70ac None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:50:59.860471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa8e89-1387-48ac-9e26-55f4c2ffb6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:59.862039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa8e89-1387-48ac-9e26-55f4c2ffb6bb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.862961 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa8e89-1387-48ac-9e26-55f4c2ffb6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.863742 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa8e89-1387-48ac-9e26-55f4c2ffb6bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.878261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa8e89-1387-48ac-9e26-55f4c2ffb6bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:50:59.888360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ea4f4-fa31-4335-8523-dca1c916bc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-62fa8e89-1387-48ac-9e26-55f4c2ffb6bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:50:59.889590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:50:59.890495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:50:59.891240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:50:59.891825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 59/122] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:50:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:50:59.892647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ea4f4-fa31-4335-8523-dca1c916bc94 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.893099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b6952-0244-4ea0-97db-25158c7a909a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:50:59.893320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ea4f4-fa31-4335-8523-dca1c916bc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.893429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b6952-0244-4ea0-97db-25158c7a909a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:50:59.893666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b6952-0244-4ea0-97db-25158c7a909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.894101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ea4f4-fa31-4335-8523-dca1c916bc94 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:50:59.895106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ea4f4-fa31-4335-8523-dca1c916bc94 None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:50:59.895606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ea4f4-fa31-4335-8523-dca1c916bc94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:50:59.897414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ea4f4-fa31-4335-8523-dca1c916bc94 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:50:59.913411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 64/123] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:50:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:50:59.917470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ea4f4-fa31-4335-8523-dca1c916bc94 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:50:59.922816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-dd6b6952-0244-4ea0-97db-25158c7a909a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:50:59.925507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dd6b6952-0244-4ea0-97db-25158c7a909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:00.020416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 60/124] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:00.020416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b6952-0244-4ea0-97db-25158c7a909a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:00.059187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ea4f4-fa31-4335-8523-dca1c916bc94 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:00.085693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:00.102119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e3c11-9d4a-4212-ab06-0054dacf19c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:00.103385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e3c11-9d4a-4212-ab06-0054dacf19c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:00.103895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e3c11-9d4a-4212-ab06-0054dacf19c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:00.105134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e3c11-9d4a-4212-ab06-0054dacf19c7 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162/users/ded448ce0e8547839095b6fee4b26e38/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:00.107605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e3c11-9d4a-4212-ab06-0054dacf19c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=faa8ebf890584746af20166b89a7e162, user_id=ded448ce0e8547839095b6fee4b26e38, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:00.109452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e3c11-9d4a-4212-ab06-0054dacf19c7 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:00.116486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:00.133119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 65/125] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:50:59 2022] PUT /identity/v3/projects/faa8ebf890584746af20166b89a7e162/users/ded448ce0e8547839095b6fee4b26e38/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:00.137623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e3c11-9d4a-4212-ab06-0054dacf19c7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:00.155157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b6952-0244-4ea0-97db-25158c7a909a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:00.203364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc0a5c-8138-4026-acfa-f0f42839fdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:00.205377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc0a5c-8138-4026-acfa-f0f42839fdd8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:00.205899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc0a5c-8138-4026-acfa-f0f42839fdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:00.206312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc0a5c-8138-4026-acfa-f0f42839fdd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:00.207262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc0a5c-8138-4026-acfa-f0f42839fdd8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:00.207707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc0a5c-8138-4026-acfa-f0f42839fdd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:00.209340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc0a5c-8138-4026-acfa-f0f42839fdd8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:00.219967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 61/126] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:00.221483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc0a5c-8138-4026-acfa-f0f42839fdd8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:00.339727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e3c11-9d4a-4212-ab06-0054dacf19c7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:00.361199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc0a5c-8138-4026-acfa-f0f42839fdd8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:00.376803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:00.392785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232b0ab-66b4-4070-9530-753ed493e5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:00.393798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232b0ab-66b4-4070-9530-753ed493e5bc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:00.394117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232b0ab-66b4-4070-9530-753ed493e5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:00.394402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232b0ab-66b4-4070-9530-753ed493e5bc None admin] PATH_INFO: `/v3/users/9dd12833b3ae4101a15287c432ec51b3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:00.395986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232b0ab-66b4-4070-9530-753ed493e5bc None admin] RBAC: Authorizing `identity:get_user(user_id=9dd12833b3ae4101a15287c432ec51b3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:00.397437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232b0ab-66b4-4070-9530-753ed493e5bc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:00.408513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 66/127] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:00 2022] GET /identity/v3/users/9dd12833b3ae4101a15287c432ec51b3 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:00.409685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232b0ab-66b4-4070-9530-753ed493e5bc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:00.410782 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f93cc0-80f9-4447-8190-0e616fc981d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:00.411868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93cc0-80f9-4447-8190-0e616fc981d7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:00.412160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93cc0-80f9-4447-8190-0e616fc981d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:00.412467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93cc0-80f9-4447-8190-0e616fc981d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:00.433543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f93cc0-80f9-4447-8190-0e616fc981d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-41f93cc0-80f9-4447-8190-0e616fc981d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:00.442076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:00.442953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:00.443818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 62/128] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:00.444476 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f93cc0-80f9-4447-8190-0e616fc981d7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:00.489181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:00.531026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232b0ab-66b4-4070-9530-753ed493e5bc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:00.549603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f93cc0-80f9-4447-8190-0e616fc981d7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:00.595876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2e5d4-3a59-485e-ab38-0d196c0b2e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:00.597597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2e5d4-3a59-485e-ab38-0d196c0b2e50 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:00.598482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2e5d4-3a59-485e-ab38-0d196c0b2e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:00.599430 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2e5d4-3a59-485e-ab38-0d196c0b2e50 None admin] PATH_INFO: `/v3/users/9dd12833b3ae4101a15287c432ec51b3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:00.601649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2e5d4-3a59-485e-ab38-0d196c0b2e50 None admin] RBAC: Authorizing `identity:get_user(user_id=9dd12833b3ae4101a15287c432ec51b3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:00.603622 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2e5d4-3a59-485e-ab38-0d196c0b2e50 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:00.606508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 67/129] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:00 2022] GET /identity/v3/users/9dd12833b3ae4101a15287c432ec51b3 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:00.607418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7066ba-2709-43c0-acb8-58571d942014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:00.608625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7066ba-2709-43c0-acb8-58571d942014 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:00.609106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7066ba-2709-43c0-acb8-58571d942014 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:00.609573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7066ba-2709-43c0-acb8-58571d942014 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:00.611588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7066ba-2709-43c0-acb8-58571d942014 None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:00.613287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7066ba-2709-43c0-acb8-58571d942014 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:00.613760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2e5d4-3a59-485e-ab38-0d196c0b2e50 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:00.616083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 63/130] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:00 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:00.617132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7066ba-2709-43c0-acb8-58571d942014 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:00.725825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2e5d4-3a59-485e-ab38-0d196c0b2e50 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:00.734070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:00.758544 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:00.765285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864de7-5783-43d2-abe8-0e4322a27d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:00.766225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864de7-5783-43d2-abe8-0e4322a27d28 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:00.766725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864de7-5783-43d2-abe8-0e4322a27d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:00.767205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864de7-5783-43d2-abe8-0e4322a27d28 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:00.768166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16864de7-5783-43d2-abe8-0e4322a27d28 None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:00.768704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16864de7-5783-43d2-abe8-0e4322a27d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:00.770300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16864de7-5783-43d2-abe8-0e4322a27d28 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:00.779863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 68/131] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:00.781124 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864de7-5783-43d2-abe8-0e4322a27d28 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:00.873123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7066ba-2709-43c0-acb8-58571d942014 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:00.938646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f99fc-6116-4acb-b316-5cb8fe70a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:00.939897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f99fc-6116-4acb-b316-5cb8fe70a7fb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:00.940376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f99fc-6116-4acb-b316-5cb8fe70a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:00.940903 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f99fc-6116-4acb-b316-5cb8fe70a7fb None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:00.942676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f99fc-6116-4acb-b316-5cb8fe70a7fb None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:00.944614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f99fc-6116-4acb-b316-5cb8fe70a7fb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:00.947130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 64/132] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:00 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:00.949006 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f99fc-6116-4acb-b316-5cb8fe70a7fb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:00.982477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864de7-5783-43d2-abe8-0e4322a27d28 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:01.066931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32812e75-8761-4d1f-a635-b2fbb5164f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:01.067948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32812e75-8761-4d1f-a635-b2fbb5164f13 None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:01.068242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32812e75-8761-4d1f-a635-b2fbb5164f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:01.068552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32812e75-8761-4d1f-a635-b2fbb5164f13 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162/users/9dd12833b3ae4101a15287c432ec51b3/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:01.070690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32812e75-8761-4d1f-a635-b2fbb5164f13 None admin] RBAC: Authorizing `identity:create_grant(project_id=faa8ebf890584746af20166b89a7e162, user_id=9dd12833b3ae4101a15287c432ec51b3, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:01.072338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32812e75-8761-4d1f-a635-b2fbb5164f13 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:01.078073 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f99fc-6116-4acb-b316-5cb8fe70a7fb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:01.090912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:01.109830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 69/133] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:51:00 2022] PUT /identity/v3/projects/faa8ebf890584746af20166b89a7e162/users/9dd12833b3ae4101a15287c432ec51b3/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:01.110653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32812e75-8761-4d1f-a635-b2fbb5164f13 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:01.119415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b894ec-80df-4a2d-b26d-a06217567df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:01.121238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b894ec-80df-4a2d-b26d-a06217567df9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:01.122794 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b894ec-80df-4a2d-b26d-a06217567df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:01.123354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b894ec-80df-4a2d-b26d-a06217567df9 None admin] PATH_INFO: `/v3/users/9dd12833b3ae4101a15287c432ec51b3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:01.125242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b894ec-80df-4a2d-b26d-a06217567df9 None admin] RBAC: Authorizing `identity:get_user(user_id=9dd12833b3ae4101a15287c432ec51b3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:01.127026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b894ec-80df-4a2d-b26d-a06217567df9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:01.129787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 65/134] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:00 2022] GET /identity/v3/users/9dd12833b3ae4101a15287c432ec51b3 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:01.135511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b894ec-80df-4a2d-b26d-a06217567df9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:01.246750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:01.308531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32812e75-8761-4d1f-a635-b2fbb5164f13 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:01.352138 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48aead3-972d-4baf-9e9c-e85d6d28c265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:01.353643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48aead3-972d-4baf-9e9c-e85d6d28c265 None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:01.354092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48aead3-972d-4baf-9e9c-e85d6d28c265 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:01.354610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48aead3-972d-4baf-9e9c-e85d6d28c265 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162/users/9dd12833b3ae4101a15287c432ec51b3/roles/c45c9440d2b8447ab50b814c6be35bf1` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:01.357263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48aead3-972d-4baf-9e9c-e85d6d28c265 None admin] RBAC: Authorizing `identity:create_grant(project_id=faa8ebf890584746af20166b89a7e162, user_id=9dd12833b3ae4101a15287c432ec51b3, role_id=c45c9440d2b8447ab50b814c6be35bf1)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:01.359790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48aead3-972d-4baf-9e9c-e85d6d28c265 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:01.379654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 70/135] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:51:01 2022] PUT /identity/v3/projects/faa8ebf890584746af20166b89a7e162/users/9dd12833b3ae4101a15287c432ec51b3/roles/c45c9440d2b8447ab50b814c6be35bf1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:01.415749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b894ec-80df-4a2d-b26d-a06217567df9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:01.446246 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732e294-7ead-42cf-a2e4-ef3e8b282496 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:01.446818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732e294-7ead-42cf-a2e4-ef3e8b282496 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:01.447274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732e294-7ead-42cf-a2e4-ef3e8b282496 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:01.452710 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 71/136] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:01.461212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137e7d-bfcf-41af-b7c5-cb60871bbe87 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:01.462359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137e7d-bfcf-41af-b7c5-cb60871bbe87 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:01.462862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137e7d-bfcf-41af-b7c5-cb60871bbe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:01.486493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3f137e7d-bfcf-41af-b7c5-cb60871bbe87 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:01.488059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3f137e7d-bfcf-41af-b7c5-cb60871bbe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:01.494208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ef003-aba8-4424-a32e-e04d20e84161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:01.495138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ef003-aba8-4424-a32e-e04d20e84161 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:01.495529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ef003-aba8-4424-a32e-e04d20e84161 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:01.496017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ef003-aba8-4424-a32e-e04d20e84161 None admin] PATH_INFO: `/v3/projects/2837e57df6a94a86a7e8d7c2ec65ee48` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:01.501389 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ef003-aba8-4424-a32e-e04d20e84161 None admin] RBAC: Authorizing `identity:get_project(project_id=2837e57df6a94a86a7e8d7c2ec65ee48)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:01.502995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ef003-aba8-4424-a32e-e04d20e84161 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:01.504996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 66/137] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:01 2022] GET /identity/v3/projects/2837e57df6a94a86a7e8d7c2ec65ee48 => generated 312 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:01.511243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ef003-aba8-4424-a32e-e04d20e84161 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:01.576775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 72/138] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:01.602258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137e7d-bfcf-41af-b7c5-cb60871bbe87 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:01.615914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:01.667694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:01.742258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ef003-aba8-4424-a32e-e04d20e84161 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:01.784634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee83534-2465-4ea0-a224-d986b6ca947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:01.785552 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee83534-2465-4ea0-a224-d986b6ca947a None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:01.785839 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee83534-2465-4ea0-a224-d986b6ca947a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:01.786115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee83534-2465-4ea0-a224-d986b6ca947a None admin] PATH_INFO: `/v3/projects/2837e57df6a94a86a7e8d7c2ec65ee48/users/9dd12833b3ae4101a15287c432ec51b3/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:01.789728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee83534-2465-4ea0-a224-d986b6ca947a None admin] RBAC: Authorizing `identity:create_grant(project_id=2837e57df6a94a86a7e8d7c2ec65ee48, user_id=9dd12833b3ae4101a15287c432ec51b3, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:01.792280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee83534-2465-4ea0-a224-d986b6ca947a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:01.804423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137e7d-bfcf-41af-b7c5-cb60871bbe87 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:01.819136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 67/139] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:51:01 2022] PUT /identity/v3/projects/2837e57df6a94a86a7e8d7c2ec65ee48/users/9dd12833b3ae4101a15287c432ec51b3/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 01:51:01.820639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9be56-edbe-4227-b29a-d09935065708 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:01.820944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9be56-edbe-4227-b29a-d09935065708 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:01.821232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9be56-edbe-4227-b29a-d09935065708 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:01.822550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 68/140] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:01.829139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af8c57-6723-4eb4-a669-4faa9dbac22f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:01.829501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af8c57-6723-4eb4-a669-4faa9dbac22f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:01.829756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af8c57-6723-4eb4-a669-4faa9dbac22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:01.847725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c440ef-4392-41de-88e0-33d4bd8149b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:01.849026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c440ef-4392-41de-88e0-33d4bd8149b3 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:01.849527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c440ef-4392-41de-88e0-33d4bd8149b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:01.850007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c440ef-4392-41de-88e0-33d4bd8149b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:01.851569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c440ef-4392-41de-88e0-33d4bd8149b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:01.853380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c440ef-4392-41de-88e0-33d4bd8149b3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:01.855374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 73/141] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:01.861048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-75af8c57-6723-4eb4-a669-4faa9dbac22f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:01.861229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c440ef-4392-41de-88e0-33d4bd8149b3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:01.862284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-75af8c57-6723-4eb4-a669-4faa9dbac22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:01.955308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 69/142] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:01.999620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af8c57-6723-4eb4-a669-4faa9dbac22f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:02.030535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c440ef-4392-41de-88e0-33d4bd8149b3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:02.046045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:02.049370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:02.104906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd333cd-36f9-4db1-a04b-cb1f03cb7999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:02.106137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd333cd-36f9-4db1-a04b-cb1f03cb7999 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:02.106626 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd333cd-36f9-4db1-a04b-cb1f03cb7999 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:02.107102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd333cd-36f9-4db1-a04b-cb1f03cb7999 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:02.108054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd333cd-36f9-4db1-a04b-cb1f03cb7999 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:02.109821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd333cd-36f9-4db1-a04b-cb1f03cb7999 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:02.138082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 74/143] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:51:01 2022] POST /identity/v3/users => generated 283 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:51:02.158709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af8c57-6723-4eb4-a669-4faa9dbac22f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:02.207478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d791fa-f8dc-43c9-9256-4469e5032c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:02.208673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d791fa-f8dc-43c9-9256-4469e5032c21 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:02.209177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d791fa-f8dc-43c9-9256-4469e5032c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:02.209645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d791fa-f8dc-43c9-9256-4469e5032c21 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:02.223515 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d791fa-f8dc-43c9-9256-4469e5032c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-15d791fa-f8dc-43c9-9256-4469e5032c21 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:02.232438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:02.233421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:02.234162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:02.237608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 70/144] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:02.243808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd333cd-36f9-4db1-a04b-cb1f03cb7999 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:02.362564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd333cd-36f9-4db1-a04b-cb1f03cb7999 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:02.408054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04018cf3-a12f-40e8-8752-e87efe1b4a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:02.408838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04018cf3-a12f-40e8-8752-e87efe1b4a5f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:02.409130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04018cf3-a12f-40e8-8752-e87efe1b4a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:02.409412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04018cf3-a12f-40e8-8752-e87efe1b4a5f None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:02.410182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04018cf3-a12f-40e8-8752-e87efe1b4a5f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:02.410492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04018cf3-a12f-40e8-8752-e87efe1b4a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:02.412273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04018cf3-a12f-40e8-8752-e87efe1b4a5f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:02.425154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 75/145] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:02.430608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d791fa-f8dc-43c9-9256-4469e5032c21 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:02.574220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:02.591653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d791fa-f8dc-43c9-9256-4469e5032c21 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:02.634740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439543d-fd35-4d6d-9dba-dc96a6180788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:02.636338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439543d-fd35-4d6d-9dba-dc96a6180788 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:02.636906 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439543d-fd35-4d6d-9dba-dc96a6180788 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:02.637394 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439543d-fd35-4d6d-9dba-dc96a6180788 None admin] PATH_INFO: `/v3/users/eaa273ae467c455db58a3326415ce7ea` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:02.638932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439543d-fd35-4d6d-9dba-dc96a6180788 None admin] RBAC: Authorizing `identity:get_user(user_id=eaa273ae467c455db58a3326415ce7ea)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:02.640656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439543d-fd35-4d6d-9dba-dc96a6180788 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:02.642843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 71/146] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:02 2022] GET /identity/v3/users/eaa273ae467c455db58a3326415ce7ea => generated 289 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:02.647867 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04018cf3-a12f-40e8-8752-e87efe1b4a5f None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:02.705439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:02.796437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04018cf3-a12f-40e8-8752-e87efe1b4a5f None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:02.833905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a98e8-ddd4-4bf0-971b-2464ae99fcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:02.834705 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a98e8-ddd4-4bf0-971b-2464ae99fcba None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:02.834987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a98e8-ddd4-4bf0-971b-2464ae99fcba None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:02.835262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a98e8-ddd4-4bf0-971b-2464ae99fcba None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:02.836506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a98e8-ddd4-4bf0-971b-2464ae99fcba None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:02.837982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a98e8-ddd4-4bf0-971b-2464ae99fcba None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:02.846409 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439543d-fd35-4d6d-9dba-dc96a6180788 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:02.846745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 76/147] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:02 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:03.007113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439543d-fd35-4d6d-9dba-dc96a6180788 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:03.044336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43c244-94e4-4f05-be21-8d1aee37ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:03.045530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43c244-94e4-4f05-be21-8d1aee37ec93 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:03.045955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43c244-94e4-4f05-be21-8d1aee37ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:03.046379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43c244-94e4-4f05-be21-8d1aee37ec93 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162/users/eaa273ae467c455db58a3326415ce7ea/roles/e9245ca6f01f444dbbd7fe5e6121449c` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:03.050690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43c244-94e4-4f05-be21-8d1aee37ec93 None admin] RBAC: Authorizing `identity:create_grant(project_id=faa8ebf890584746af20166b89a7e162, user_id=eaa273ae467c455db58a3326415ce7ea, role_id=e9245ca6f01f444dbbd7fe5e6121449c)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:03.052320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43c244-94e4-4f05-be21-8d1aee37ec93 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:03.072249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 72/148] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:51:02 2022] PUT /identity/v3/projects/faa8ebf890584746af20166b89a7e162/users/eaa273ae467c455db58a3326415ce7ea/roles/e9245ca6f01f444dbbd7fe5e6121449c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 01:51:03.489641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358434-6517-4769-baec-1452a1ab847e None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:03.490157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358434-6517-4769-baec-1452a1ab847e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:03.490367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358434-6517-4769-baec-1452a1ab847e None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:03.491923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 77/149] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:03.500736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484e9c7-53b5-494c-b752-5e9e7c2d5d26 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:03.501641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484e9c7-53b5-494c-b752-5e9e7c2d5d26 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:03.502111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484e9c7-53b5-494c-b752-5e9e7c2d5d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:03.520969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4484e9c7-53b5-494c-b752-5e9e7c2d5d26 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:03.522481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4484e9c7-53b5-494c-b752-5e9e7c2d5d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:03.541434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:03.603300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 73/150] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:03.640849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63358434-6517-4769-baec-1452a1ab847e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:03.759015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:03.769089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63358434-6517-4769-baec-1452a1ab847e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:03.822468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f1120-3c81-42d0-8fcd-2fb54a6fbfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:03.824301 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f1120-3c81-42d0-8fcd-2fb54a6fbfaa None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:03.824865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f1120-3c81-42d0-8fcd-2fb54a6fbfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:03.825352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f1120-3c81-42d0-8fcd-2fb54a6fbfaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:03.839860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f1120-3c81-42d0-8fcd-2fb54a6fbfaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-3f3f1120-3c81-42d0-8fcd-2fb54a6fbfaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:03.848008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:03.848975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:03.849740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:03.852116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 78/151] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:51:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:03.858796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484e9c7-53b5-494c-b752-5e9e7c2d5d26 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:03.998684 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484e9c7-53b5-494c-b752-5e9e7c2d5d26 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:04.039904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408c378-7bb9-4b03-94b2-af860bc6e385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:04.040588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c378-7bb9-4b03-94b2-af860bc6e385 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:04.040870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c378-7bb9-4b03-94b2-af860bc6e385 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:04.041145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c378-7bb9-4b03-94b2-af860bc6e385 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:04.042000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408c378-7bb9-4b03-94b2-af860bc6e385 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:04.042319 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408c378-7bb9-4b03-94b2-af860bc6e385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:04.043703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408c378-7bb9-4b03-94b2-af860bc6e385 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:04.055305 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 74/152] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:04.061391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f1120-3c81-42d0-8fcd-2fb54a6fbfaa None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:04.202708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f1120-3c81-42d0-8fcd-2fb54a6fbfaa None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:04.252009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b572d-af9f-4757-9829-f9bd136ec91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:04.253316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b572d-af9f-4757-9829-f9bd136ec91b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:04.253802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b572d-af9f-4757-9829-f9bd136ec91b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:04.254277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b572d-af9f-4757-9829-f9bd136ec91b None admin] PATH_INFO: `/v3/users/ded448ce0e8547839095b6fee4b26e38` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:04.261386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b572d-af9f-4757-9829-f9bd136ec91b None admin] RBAC: Authorizing `identity:get_user(user_id=ded448ce0e8547839095b6fee4b26e38)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:04.263178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b572d-af9f-4757-9829-f9bd136ec91b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:04.266384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 79/153] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:04 2022] GET /identity/v3/users/ded448ce0e8547839095b6fee4b26e38 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:04.271760 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408c378-7bb9-4b03-94b2-af860bc6e385 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:04.293935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:04.406878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408c378-7bb9-4b03-94b2-af860bc6e385 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:04.447126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6198b-54ac-4963-b2a3-35308215538a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:04.448302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6198b-54ac-4963-b2a3-35308215538a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:04.448624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6198b-54ac-4963-b2a3-35308215538a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:04.448898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6198b-54ac-4963-b2a3-35308215538a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:04.526693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6198b-54ac-4963-b2a3-35308215538a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:04.528259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b6198b-54ac-4963-b2a3-35308215538a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:04.540557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:04.654709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 75/154] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:04.656039 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b572d-af9f-4757-9829-f9bd136ec91b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:04.709832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:04.818961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b572d-af9f-4757-9829-f9bd136ec91b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:04.867805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8704c-1bff-46d2-a481-44a196642b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:04.868706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8704c-1bff-46d2-a481-44a196642b34 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:04.868986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8704c-1bff-46d2-a481-44a196642b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:04.869259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8704c-1bff-46d2-a481-44a196642b34 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:04.873183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8704c-1bff-46d2-a481-44a196642b34 None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:04.874649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8704c-1bff-46d2-a481-44a196642b34 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:04.876365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 80/155] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:04 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:04.880159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a5ec4-eea4-4b93-aed8-72e1a3f4f45c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:04.880470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a5ec4-eea4-4b93-aed8-72e1a3f4f45c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:04.880748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a5ec4-eea4-4b93-aed8-72e1a3f4f45c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:04.882193 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 76/156] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:04.883106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8704c-1bff-46d2-a481-44a196642b34 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:04.899115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea285c3-6840-4cf1-a5cc-6dd8e4cbff44 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:04.899406 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea285c3-6840-4cf1-a5cc-6dd8e4cbff44 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:04.899667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea285c3-6840-4cf1-a5cc-6dd8e4cbff44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:04.932056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3ea285c3-6840-4cf1-a5cc-6dd8e4cbff44 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:04.938530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3ea285c3-6840-4cf1-a5cc-6dd8e4cbff44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:05.044074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 77/157] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:05.084190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea285c3-6840-4cf1-a5cc-6dd8e4cbff44 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:05.154942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:05.190090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:05.204096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8704c-1bff-46d2-a481-44a196642b34 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:05.267124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea285c3-6840-4cf1-a5cc-6dd8e4cbff44 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:05.294218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386d707-c8f3-4019-a03d-52f8ca9d78b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:05.295169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386d707-c8f3-4019-a03d-52f8ca9d78b8 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:05.295593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386d707-c8f3-4019-a03d-52f8ca9d78b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:05.296005 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386d707-c8f3-4019-a03d-52f8ca9d78b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:05.297018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386d707-c8f3-4019-a03d-52f8ca9d78b8 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=faa8ebf890584746af20166b89a7e162, user.id=ded448ce0e8547839095b6fee4b26e38) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:05.297485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386d707-c8f3-4019-a03d-52f8ca9d78b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:05.299076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386d707-c8f3-4019-a03d-52f8ca9d78b8 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:05.312040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 81/158] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:04 2022] GET /identity/v3/role_assignments?user.id=ded448ce0e8547839095b6fee4b26e38&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 608 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:05.338057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227c395-2ea9-4041-a344-31cccc5b85f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:05.339038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227c395-2ea9-4041-a344-31cccc5b85f5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:05.339388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227c395-2ea9-4041-a344-31cccc5b85f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:05.339673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227c395-2ea9-4041-a344-31cccc5b85f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:05.364619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227c395-2ea9-4041-a344-31cccc5b85f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5227c395-2ea9-4041-a344-31cccc5b85f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:05.373042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:05.373900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:05.374711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 78/159] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:05.381309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386d707-c8f3-4019-a03d-52f8ca9d78b8 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:05.521998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386d707-c8f3-4019-a03d-52f8ca9d78b8 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:05.568339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda61c1b-ab28-4680-97c5-6619799559d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:05.569058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda61c1b-ab28-4680-97c5-6619799559d5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:05.569340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda61c1b-ab28-4680-97c5-6619799559d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:05.569613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda61c1b-ab28-4680-97c5-6619799559d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:05.570374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda61c1b-ab28-4680-97c5-6619799559d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:05.570683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda61c1b-ab28-4680-97c5-6619799559d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:05.572062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda61c1b-ab28-4680-97c5-6619799559d5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:05.582600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 82/160] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:05.588058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227c395-2ea9-4041-a344-31cccc5b85f5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:05.735052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:05.739204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227c395-2ea9-4041-a344-31cccc5b85f5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:05.748658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab063908-17de-4121-b6f9-8f05ed1d0216 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:05.749677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab063908-17de-4121-b6f9-8f05ed1d0216 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:05.750151 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab063908-17de-4121-b6f9-8f05ed1d0216 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:05.751733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 83/161] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:51:05.758408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f28eb6-e053-4e39-b382-2d4902cbeb30 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:05.758991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f28eb6-e053-4e39-b382-2d4902cbeb30 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:05.759460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f28eb6-e053-4e39-b382-2d4902cbeb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:05.790243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f65fe-6aa8-4ac2-a289-b5a7575a08f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:05.791064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f65fe-6aa8-4ac2-a289-b5a7575a08f8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:05.791306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-17f28eb6-e053-4e39-b382-2d4902cbeb30 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:05.791426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f65fe-6aa8-4ac2-a289-b5a7575a08f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:05.791692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f65fe-6aa8-4ac2-a289-b5a7575a08f8 None admin] PATH_INFO: `/v3/users/9dd12833b3ae4101a15287c432ec51b3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:05.793030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f65fe-6aa8-4ac2-a289-b5a7575a08f8 None admin] RBAC: Authorizing `identity:get_user(user_id=9dd12833b3ae4101a15287c432ec51b3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:05.793663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-17f28eb6-e053-4e39-b382-2d4902cbeb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:05.794545 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f65fe-6aa8-4ac2-a289-b5a7575a08f8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:05.799630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 79/162] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:05 2022] GET /identity/v3/users/9dd12833b3ae4101a15287c432ec51b3 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:05.802381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f65fe-6aa8-4ac2-a289-b5a7575a08f8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:05.821721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:05.929428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 84/163] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:05.930778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1dab0-66cf-4853-9fae-0a666ebaeea0 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:05.931065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1dab0-66cf-4853-9fae-0a666ebaeea0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:05.931342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1dab0-66cf-4853-9fae-0a666ebaeea0 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:05.932887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 85/164] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:05.950178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3be8fa-bb77-4c6f-87a0-fa55b44952bc None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:05.950473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3be8fa-bb77-4c6f-87a0-fa55b44952bc None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:05.950738 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3be8fa-bb77-4c6f-87a0-fa55b44952bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:05.974036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-0a3be8fa-bb77-4c6f-87a0-fa55b44952bc None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:05.975241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0a3be8fa-bb77-4c6f-87a0-fa55b44952bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:06.006281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f65fe-6aa8-4ac2-a289-b5a7575a08f8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:06.066930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bd1cd-d171-49f6-b8c4-c3d18471fd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:06.067526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bd1cd-d171-49f6-b8c4-c3d18471fd41 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:06.067809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bd1cd-d171-49f6-b8c4-c3d18471fd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:06.068081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bd1cd-d171-49f6-b8c4-c3d18471fd41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:06.103779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 86/165] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:06.105806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3be8fa-bb77-4c6f-87a0-fa55b44952bc None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:06.144780 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:06.221298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3bd1cd-d171-49f6-b8c4-c3d18471fd41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:06.222796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3bd1cd-d171-49f6-b8c4-c3d18471fd41 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:06.223326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:06.248921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3be8fa-bb77-4c6f-87a0-fa55b44952bc None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:06.323491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682b691-d97a-45bd-9156-1391470f699c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:06.324408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682b691-d97a-45bd-9156-1391470f699c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:06.324700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682b691-d97a-45bd-9156-1391470f699c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:06.324976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682b691-d97a-45bd-9156-1391470f699c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:06.349109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682b691-d97a-45bd-9156-1391470f699c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-d682b691-d97a-45bd-9156-1391470f699c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:06.363672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:06.364694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:06.365701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 87/166] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:06.366354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682b691-d97a-45bd-9156-1391470f699c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:06.427490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 80/167] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 08 01:51:06.428662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bd1cd-d171-49f6-b8c4-c3d18471fd41 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:06.592823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:06.602540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682b691-d97a-45bd-9156-1391470f699c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:06.662259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cb883-b60c-4c28-af67-658dc44cdd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:06.663063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cb883-b60c-4c28-af67-658dc44cdd32 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:06.663344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cb883-b60c-4c28-af67-658dc44cdd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:06.663615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cb883-b60c-4c28-af67-658dc44cdd32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:06.678747 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008cb883-b60c-4c28-af67-658dc44cdd32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:06.686504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bd1cd-d171-49f6-b8c4-c3d18471fd41 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-008cb883-b60c-4c28-af67-658dc44cdd32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:06.687528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:06.688389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:06.689210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 88/168] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:06.689869 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cb883-b60c-4c28-af67-658dc44cdd32 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:06.694573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:06.795251 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cb883-b60c-4c28-af67-658dc44cdd32 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:06.817117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac172305-70ea-4665-aa40-e5a5271378e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:06.817807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac172305-70ea-4665-aa40-e5a5271378e8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:06.818091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac172305-70ea-4665-aa40-e5a5271378e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:06.818368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac172305-70ea-4665-aa40-e5a5271378e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:06.819127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac172305-70ea-4665-aa40-e5a5271378e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:06.819432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac172305-70ea-4665-aa40-e5a5271378e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:06.820848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac172305-70ea-4665-aa40-e5a5271378e8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:06.841156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 81/169] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:06.841844 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac172305-70ea-4665-aa40-e5a5271378e8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:06.846207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea34df-6ebd-4842-b487-48411ffbe43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:06.847024 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea34df-6ebd-4842-b487-48411ffbe43c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:06.847299 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea34df-6ebd-4842-b487-48411ffbe43c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:06.847574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea34df-6ebd-4842-b487-48411ffbe43c None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:06.849340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea34df-6ebd-4842-b487-48411ffbe43c None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:06.857814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea34df-6ebd-4842-b487-48411ffbe43c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:06.859917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 89/170] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:06 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:06.863734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea34df-6ebd-4842-b487-48411ffbe43c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:07.037002 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:07.086030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea34df-6ebd-4842-b487-48411ffbe43c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:07.115772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac172305-70ea-4665-aa40-e5a5271378e8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:07.145113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bbfe5-1b05-43ad-a346-1e7dde4fdb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:07.146073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bbfe5-1b05-43ad-a346-1e7dde4fdb33 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.146352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bbfe5-1b05-43ad-a346-1e7dde4fdb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.146637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bbfe5-1b05-43ad-a346-1e7dde4fdb33 None admin] PATH_INFO: `/v3/users/9dd12833b3ae4101a15287c432ec51b3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:07.147952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:07.149003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bbfe5-1b05-43ad-a346-1e7dde4fdb33 None admin] RBAC: Authorizing `identity:get_user(user_id=9dd12833b3ae4101a15287c432ec51b3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:07.150726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bbfe5-1b05-43ad-a346-1e7dde4fdb33 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:07.154159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 90/171] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:06 2022] GET /identity/v3/users/9dd12833b3ae4101a15287c432ec51b3 => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:07.155252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bbfe5-1b05-43ad-a346-1e7dde4fdb33 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:07.177127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5658079-0af1-4c36-8d92-9022f5bac141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:07.177825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5658079-0af1-4c36-8d92-9022f5bac141 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.178111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5658079-0af1-4c36-8d92-9022f5bac141 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.178386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5658079-0af1-4c36-8d92-9022f5bac141 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:07.179143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5658079-0af1-4c36-8d92-9022f5bac141 None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:07.179449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5658079-0af1-4c36-8d92-9022f5bac141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:07.180879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5658079-0af1-4c36-8d92-9022f5bac141 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:07.201685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 82/172] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:07.203710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5587dd-cfcb-44a6-9c76-39f05a82805d None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.204004 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5587dd-cfcb-44a6-9c76-39f05a82805d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.204282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5587dd-cfcb-44a6-9c76-39f05a82805d None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:07.212007 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 83/173] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:07.217533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5587dd-cfcb-44a6-9c76-39f05a82805d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:07.315376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bbfe5-1b05-43ad-a346-1e7dde4fdb33 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:07.357541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf080f-8341-4b8c-ab9a-322e85f23566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:07.358583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf080f-8341-4b8c-ab9a-322e85f23566 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.359072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf080f-8341-4b8c-ab9a-322e85f23566 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.359568 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf080f-8341-4b8c-ab9a-322e85f23566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:07.360654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf080f-8341-4b8c-ab9a-322e85f23566 None admin] RBAC: Adding query filter params (role.id=61490cf1d7334421b84ff67552b79bfc, scope.project.id=faa8ebf890584746af20166b89a7e162, user.id=9dd12833b3ae4101a15287c432ec51b3) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:07.361313 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf080f-8341-4b8c-ab9a-322e85f23566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:07.363517 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf080f-8341-4b8c-ab9a-322e85f23566 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:07.371981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 91/174] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:07 2022] GET /identity/v3/role_assignments?user.id=9dd12833b3ae4101a15287c432ec51b3&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=61490cf1d7334421b84ff67552b79bfc => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:07.376811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:07.378004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf080f-8341-4b8c-ab9a-322e85f23566 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:07.574630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf080f-8341-4b8c-ab9a-322e85f23566 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:07.576699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5587dd-cfcb-44a6-9c76-39f05a82805d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:07.646323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7f7bf-d3c2-4cfd-b8d8-1324ae4ece14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:07.646923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7f7bf-d3c2-4cfd-b8d8-1324ae4ece14 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.647189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7f7bf-d3c2-4cfd-b8d8-1324ae4ece14 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.647458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7f7bf-d3c2-4cfd-b8d8-1324ae4ece14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:07.653965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6396d68-4a50-4278-b27b-716b81c0d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:07.654989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6396d68-4a50-4278-b27b-716b81c0d487 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.655275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6396d68-4a50-4278-b27b-716b81c0d487 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.655551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6396d68-4a50-4278-b27b-716b81c0d487 None admin] PATH_INFO: `/v3/users/9dd12833b3ae4101a15287c432ec51b3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:07.656813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6396d68-4a50-4278-b27b-716b81c0d487 None admin] RBAC: Authorizing `identity:get_user(user_id=9dd12833b3ae4101a15287c432ec51b3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:07.658305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6396d68-4a50-4278-b27b-716b81c0d487 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:07.659985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 92/175] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:07 2022] GET /identity/v3/users/9dd12833b3ae4101a15287c432ec51b3 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:07.661224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b975d-95cf-45b3-8894-eb1514b7d51c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.661777 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b975d-95cf-45b3-8894-eb1514b7d51c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.662057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b975d-95cf-45b3-8894-eb1514b7d51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:07.697096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-c88b975d-95cf-45b3-8894-eb1514b7d51c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:07.699032 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c88b975d-95cf-45b3-8894-eb1514b7d51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:07.763920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:07.775240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7f7bf-d3c2-4cfd-b8d8-1324ae4ece14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:07.777174 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7f7bf-d3c2-4cfd-b8d8-1324ae4ece14 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:07.784821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:07.792329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 93/176] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:07.793544 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcab13-f63f-41c2-a93a-0c9317ce230f None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.793860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcab13-f63f-41c2-a93a-0c9317ce230f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.794097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcab13-f63f-41c2-a93a-0c9317ce230f None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:07.795378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 94/177] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:07.796082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcab13-f63f-41c2-a93a-0c9317ce230f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:07.919542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 84/178] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:07.919542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be5161-67b8-4e43-a718-1eb134ff8b59 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.919542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be5161-67b8-4e43-a718-1eb134ff8b59 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.919542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be5161-67b8-4e43-a718-1eb134ff8b59 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:07.919542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 85/179] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:07.919542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859619-f18b-412d-895c-fa9838042023 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.919542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859619-f18b-412d-895c-fa9838042023 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.921243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859619-f18b-412d-895c-fa9838042023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:07.930649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcab13-f63f-41c2-a93a-0c9317ce230f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:07.936874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-da859619-f18b-412d-895c-fa9838042023 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:07.938062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-da859619-f18b-412d-895c-fa9838042023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:07.964017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab770de-2d3c-4455-bbed-e0cf76d954ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:07.964789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab770de-2d3c-4455-bbed-e0cf76d954ba None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:07.965074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab770de-2d3c-4455-bbed-e0cf76d954ba None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:07.965351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab770de-2d3c-4455-bbed-e0cf76d954ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.005938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 86/180] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:08.006783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859619-f18b-412d-895c-fa9838042023 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:08.031435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:08.031887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:08.035427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab770de-2d3c-4455-bbed-e0cf76d954ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:08.037546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab770de-2d3c-4455-bbed-e0cf76d954ba None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:08.107919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859619-f18b-412d-895c-fa9838042023 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:08.135020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 95/181] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:08.136615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acfa33-ffa1-423b-9875-f2d5da41a3d7 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.136899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acfa33-ffa1-423b-9875-f2d5da41a3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.137172 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acfa33-ffa1-423b-9875-f2d5da41a3d7 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.138443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 96/182] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:08.139130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acfa33-ffa1-423b-9875-f2d5da41a3d7 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:08.147805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fa557-1b4a-4aee-af37-a7bf1edd7299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:08.148645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fa557-1b4a-4aee-af37-a7bf1edd7299 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.148931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fa557-1b4a-4aee-af37-a7bf1edd7299 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.149205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fa557-1b4a-4aee-af37-a7bf1edd7299 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.159368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fa557-1b4a-4aee-af37-a7bf1edd7299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b72fa557-1b4a-4aee-af37-a7bf1edd7299 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:08.167777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:08.168888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 87/183] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:08.174698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70366ce7-02d3-4ff1-b6ac-167cb99becf4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.175547 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70366ce7-02d3-4ff1-b6ac-167cb99becf4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.175547 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70366ce7-02d3-4ff1-b6ac-167cb99becf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.187089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-70366ce7-02d3-4ff1-b6ac-167cb99becf4 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:08.188286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-70366ce7-02d3-4ff1-b6ac-167cb99becf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:08.251975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:08.254542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 88/184] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:08.255402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70366ce7-02d3-4ff1-b6ac-167cb99becf4 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:08.268911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:08.270739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acfa33-ffa1-423b-9875-f2d5da41a3d7 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:08.306956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502312f2-65a6-4048-87f0-5ad403b1b18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:08.308435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502312f2-65a6-4048-87f0-5ad403b1b18a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.308736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502312f2-65a6-4048-87f0-5ad403b1b18a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.309007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502312f2-65a6-4048-87f0-5ad403b1b18a None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.310183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502312f2-65a6-4048-87f0-5ad403b1b18a None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:08.311632 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502312f2-65a6-4048-87f0-5ad403b1b18a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:08.313340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 97/185] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:08 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:08.314093 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502312f2-65a6-4048-87f0-5ad403b1b18a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:08.355112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70366ce7-02d3-4ff1-b6ac-167cb99becf4 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:08.389437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff6e73-8210-4c16-bfa1-a71a63495cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:08.390229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff6e73-8210-4c16-bfa1-a71a63495cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.390511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff6e73-8210-4c16-bfa1-a71a63495cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.390785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff6e73-8210-4c16-bfa1-a71a63495cc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.391940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff6e73-8210-4c16-bfa1-a71a63495cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:08.393431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff6e73-8210-4c16-bfa1-a71a63495cc8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:08.395030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 89/186] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:08.396238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed0eaa-bf44-4f52-b0ed-cbfd0943ae40 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.396553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed0eaa-bf44-4f52-b0ed-cbfd0943ae40 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.396832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed0eaa-bf44-4f52-b0ed-cbfd0943ae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.412983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-14ed0eaa-bf44-4f52-b0ed-cbfd0943ae40 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:08.414164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-14ed0eaa-bf44-4f52-b0ed-cbfd0943ae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:08.415117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502312f2-65a6-4048-87f0-5ad403b1b18a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:08.448901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc4ad5-195a-4ed2-84ab-4b775c217380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:08.449694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc4ad5-195a-4ed2-84ab-4b775c217380 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.449979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc4ad5-195a-4ed2-84ab-4b775c217380 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.450255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc4ad5-195a-4ed2-84ab-4b775c217380 None admin] PATH_INFO: `/v3/projects/2837e57df6a94a86a7e8d7c2ec65ee48` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.451439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc4ad5-195a-4ed2-84ab-4b775c217380 None admin] RBAC: Authorizing `identity:get_project(project_id=2837e57df6a94a86a7e8d7c2ec65ee48)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:08.452928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc4ad5-195a-4ed2-84ab-4b775c217380 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:08.454619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 98/187] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:08 2022] GET /identity/v3/projects/2837e57df6a94a86a7e8d7c2ec65ee48 => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:08.455372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc4ad5-195a-4ed2-84ab-4b775c217380 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:08.489181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 90/188] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:08.489181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed0eaa-bf44-4f52-b0ed-cbfd0943ae40 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:08.490147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:08.499869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:08.558004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc4ad5-195a-4ed2-84ab-4b775c217380 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:08.586907 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed0eaa-bf44-4f52-b0ed-cbfd0943ae40 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:08.592594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372346d-ab8b-4119-b3bf-47fdecc0ad2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:08.593261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372346d-ab8b-4119-b3bf-47fdecc0ad2a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.593539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372346d-ab8b-4119-b3bf-47fdecc0ad2a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.593808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372346d-ab8b-4119-b3bf-47fdecc0ad2a None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.594551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372346d-ab8b-4119-b3bf-47fdecc0ad2a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:08.594863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372346d-ab8b-4119-b3bf-47fdecc0ad2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:08.596241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372346d-ab8b-4119-b3bf-47fdecc0ad2a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:08.609446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 99/189] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:08.609446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372346d-ab8b-4119-b3bf-47fdecc0ad2a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:08.623057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a6e1d-dddc-4a79-ae7b-f68f77dd95e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:08.623876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a6e1d-dddc-4a79-ae7b-f68f77dd95e8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.624155 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a6e1d-dddc-4a79-ae7b-f68f77dd95e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.624451 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a6e1d-dddc-4a79-ae7b-f68f77dd95e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.634640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a6e1d-dddc-4a79-ae7b-f68f77dd95e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2a0a6e1d-dddc-4a79-ae7b-f68f77dd95e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:08.642865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:08.643928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 91/190] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:08.650080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a6e1d-dddc-4a79-ae7b-f68f77dd95e8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:08.702074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:08.708488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372346d-ab8b-4119-b3bf-47fdecc0ad2a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:08.715825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:08.742003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d27bc-f679-42d6-9d1e-029eba1f2758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:08.742663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d27bc-f679-42d6-9d1e-029eba1f2758 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.742943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d27bc-f679-42d6-9d1e-029eba1f2758 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.743221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d27bc-f679-42d6-9d1e-029eba1f2758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.744050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d27bc-f679-42d6-9d1e-029eba1f2758 None admin] RBAC: Adding query filter params (role.id=c45c9440d2b8447ab50b814c6be35bf1, scope.project.id=faa8ebf890584746af20166b89a7e162, user.id=9dd12833b3ae4101a15287c432ec51b3) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:08.744364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d27bc-f679-42d6-9d1e-029eba1f2758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:08.746448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d27bc-f679-42d6-9d1e-029eba1f2758 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:08.747056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a6e1d-dddc-4a79-ae7b-f68f77dd95e8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:08.752167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 100/191] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:08 2022] GET /identity/v3/role_assignments?user.id=9dd12833b3ae4101a15287c432ec51b3&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=c45c9440d2b8447ab50b814c6be35bf1 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:08.752981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d27bc-f679-42d6-9d1e-029eba1f2758 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:08.788619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37fcc3-2ce3-4253-af46-24be4e8726a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:08.789328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37fcc3-2ce3-4253-af46-24be4e8726a0 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.789608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37fcc3-2ce3-4253-af46-24be4e8726a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.789971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37fcc3-2ce3-4253-af46-24be4e8726a0 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.790707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37fcc3-2ce3-4253-af46-24be4e8726a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:08.792913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37fcc3-2ce3-4253-af46-24be4e8726a0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:08.821801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 92/192] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:51:08 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:51:08.823022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37fcc3-2ce3-4253-af46-24be4e8726a0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:08.908886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d27bc-f679-42d6-9d1e-029eba1f2758 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:08.956639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37fcc3-2ce3-4253-af46-24be4e8726a0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:08.978804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce979c9-e2d6-46a3-93da-92896b765b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:08.979980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce979c9-e2d6-46a3-93da-92896b765b52 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:08.980268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce979c9-e2d6-46a3-93da-92896b765b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:08.980580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce979c9-e2d6-46a3-93da-92896b765b52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:08.981433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce979c9-e2d6-46a3-93da-92896b765b52 None admin] RBAC: Adding query filter params (role.id=61490cf1d7334421b84ff67552b79bfc, scope.project.id=2837e57df6a94a86a7e8d7c2ec65ee48, user.id=9dd12833b3ae4101a15287c432ec51b3) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:08.981748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce979c9-e2d6-46a3-93da-92896b765b52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:08.983229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce979c9-e2d6-46a3-93da-92896b765b52 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:08.990231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 101/193] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:08 2022] GET /identity/v3/role_assignments?user.id=9dd12833b3ae4101a15287c432ec51b3&scope.project.id=2837e57df6a94a86a7e8d7c2ec65ee48&role.id=61490cf1d7334421b84ff67552b79bfc => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:08.993376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce979c9-e2d6-46a3-93da-92896b765b52 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:09.000923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:09.018241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c1ab6-8089-4647-99cd-5fa128d0b52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:09.019642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c1ab6-8089-4647-99cd-5fa128d0b52f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:09.020120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c1ab6-8089-4647-99cd-5fa128d0b52f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:09.020856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c1ab6-8089-4647-99cd-5fa128d0b52f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:09.032151 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:09.044341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44c1ab6-8089-4647-99cd-5fa128d0b52f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f44c1ab6-8089-4647-99cd-5fa128d0b52f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:09.052338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:09.053335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:09.054140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 93/194] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:51:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:09.054808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c1ab6-8089-4647-99cd-5fa128d0b52f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:09.178135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce979c9-e2d6-46a3-93da-92896b765b52 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:09.209399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44c1ab6-8089-4647-99cd-5fa128d0b52f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:09.219909 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705d1bd-12cf-44cb-b621-da4435c35fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:09.221022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d1bd-12cf-44cb-b621-da4435c35fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:09.221308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d1bd-12cf-44cb-b621-da4435c35fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:09.221582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d1bd-12cf-44cb-b621-da4435c35fa8 None admin] PATH_INFO: `/v3/users/eaa273ae467c455db58a3326415ce7ea` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:09.222791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705d1bd-12cf-44cb-b621-da4435c35fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=eaa273ae467c455db58a3326415ce7ea)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:09.224516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705d1bd-12cf-44cb-b621-da4435c35fa8 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:09.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 102/195] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:08 2022] GET /identity/v3/users/eaa273ae467c455db58a3326415ce7ea => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:09.227611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705d1bd-12cf-44cb-b621-da4435c35fa8 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:09.256973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a57df8-45d8-426e-9d64-000fcbcf794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:09.257639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a57df8-45d8-426e-9d64-000fcbcf794d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:09.257917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a57df8-45d8-426e-9d64-000fcbcf794d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:09.258191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a57df8-45d8-426e-9d64-000fcbcf794d None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:09.258947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a57df8-45d8-426e-9d64-000fcbcf794d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:09.259251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a57df8-45d8-426e-9d64-000fcbcf794d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:09.260762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a57df8-45d8-426e-9d64-000fcbcf794d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:09.271341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 94/196] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:09.272000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a57df8-45d8-426e-9d64-000fcbcf794d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:09.343722 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:09.357562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:09.369150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705d1bd-12cf-44cb-b621-da4435c35fa8 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:09.412649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a57df8-45d8-426e-9d64-000fcbcf794d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:09.416274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0693e-455a-451f-96a5-f4d30a05beb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:09.417284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0693e-455a-451f-96a5-f4d30a05beb9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:09.417721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0693e-455a-451f-96a5-f4d30a05beb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:09.418146 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0693e-455a-451f-96a5-f4d30a05beb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:09.419109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0693e-455a-451f-96a5-f4d30a05beb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:09.419586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0693e-455a-451f-96a5-f4d30a05beb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:09.421720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0693e-455a-451f-96a5-f4d30a05beb9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:09.434734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 103/197] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:09.436111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0693e-455a-451f-96a5-f4d30a05beb9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:09.455986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb07ef-b79c-4e67-bb94-820ab665d208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:09.457047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb07ef-b79c-4e67-bb94-820ab665d208 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:09.457502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb07ef-b79c-4e67-bb94-820ab665d208 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:09.457944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb07ef-b79c-4e67-bb94-820ab665d208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:09.534096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb07ef-b79c-4e67-bb94-820ab665d208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:09.536061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb07ef-b79c-4e67-bb94-820ab665d208 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:09.544872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0693e-455a-451f-96a5-f4d30a05beb9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:09.581423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67cb9f4-779c-48ce-b1fe-98ca1d6bb367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:09.582464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67cb9f4-779c-48ce-b1fe-98ca1d6bb367 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:09.582847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67cb9f4-779c-48ce-b1fe-98ca1d6bb367 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:09.583225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67cb9f4-779c-48ce-b1fe-98ca1d6bb367 None admin] PATH_INFO: `/v3/users/70c3a8b8bbd04186bf178525ae8fb4af` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:09.598054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:09.599237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67cb9f4-779c-48ce-b1fe-98ca1d6bb367 None admin] RBAC: Authorizing `identity:get_user(user_id=70c3a8b8bbd04186bf178525ae8fb4af)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:09.600677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67cb9f4-779c-48ce-b1fe-98ca1d6bb367 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:09.602398 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 104/198] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:09 2022] GET /identity/v3/users/70c3a8b8bbd04186bf178525ae8fb4af => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:09.603163 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67cb9f4-779c-48ce-b1fe-98ca1d6bb367 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:09.626703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:09.670966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 95/199] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:09.670966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb07ef-b79c-4e67-bb94-820ab665d208 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:09.753745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67cb9f4-779c-48ce-b1fe-98ca1d6bb367 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:09.772610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb07ef-b79c-4e67-bb94-820ab665d208 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:09.793867 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33728b-4323-453e-94bc-f156e72f56b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:09.794977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33728b-4323-453e-94bc-f156e72f56b7 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:09.795409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33728b-4323-453e-94bc-f156e72f56b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:09.795831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33728b-4323-453e-94bc-f156e72f56b7 None admin] PATH_INFO: `/v3/users/70c3a8b8bbd04186bf178525ae8fb4af` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:09.797277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33728b-4323-453e-94bc-f156e72f56b7 None admin] RBAC: Authorizing `identity:get_user(user_id=70c3a8b8bbd04186bf178525ae8fb4af)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:09.798945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33728b-4323-453e-94bc-f156e72f56b7 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:09.800961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 105/200] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:09 2022] GET /identity/v3/users/70c3a8b8bbd04186bf178525ae8fb4af => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:09.801925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33728b-4323-453e-94bc-f156e72f56b7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:09.815276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b54e4a-8a92-4c41-915f-b539f4060ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:09.815936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b54e4a-8a92-4c41-915f-b539f4060ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:09.816220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b54e4a-8a92-4c41-915f-b539f4060ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:09.816524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b54e4a-8a92-4c41-915f-b539f4060ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:09.836132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:09.890356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b54e4a-8a92-4c41-915f-b539f4060ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:09.891534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:09.891859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b54e4a-8a92-4c41-915f-b539f4060ec7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:09.914954 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33728b-4323-453e-94bc-f156e72f56b7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:09.953204 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a8bc-c3cc-4570-8aff-0393f6e8e365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:09.954434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a8bc-c3cc-4570-8aff-0393f6e8e365 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:09.954921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a8bc-c3cc-4570-8aff-0393f6e8e365 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:09.955423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a8bc-c3cc-4570-8aff-0393f6e8e365 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:09.956842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1a8bc-c3cc-4570-8aff-0393f6e8e365 None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:09.958503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1a8bc-c3cc-4570-8aff-0393f6e8e365 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:09.960449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 106/201] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:09 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:09.966062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a8bc-c3cc-4570-8aff-0393f6e8e365 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:10.004163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 96/202] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:10.009666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b54e4a-8a92-4c41-915f-b539f4060ec7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:10.059792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:10.075438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a8bc-c3cc-4570-8aff-0393f6e8e365 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:10.113145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b54e4a-8a92-4c41-915f-b539f4060ec7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:10.114847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee06a48-c27c-48ca-9515-ba61e3364e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:10.115859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee06a48-c27c-48ca-9515-ba61e3364e7f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:10.116343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee06a48-c27c-48ca-9515-ba61e3364e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:10.116869 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee06a48-c27c-48ca-9515-ba61e3364e7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:10.137022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:10.149514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b211f-9838-4736-8efe-978e8ad4da6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:10.150338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b211f-9838-4736-8efe-978e8ad4da6d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:10.150557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b211f-9838-4736-8efe-978e8ad4da6d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:10.151367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b211f-9838-4736-8efe-978e8ad4da6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:10.152213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b211f-9838-4736-8efe-978e8ad4da6d None admin] RBAC: Adding query filter params (role.id=e9245ca6f01f444dbbd7fe5e6121449c, scope.project.id=faa8ebf890584746af20166b89a7e162, user.id=eaa273ae467c455db58a3326415ce7ea) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:10.152557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b211f-9838-4736-8efe-978e8ad4da6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:10.154081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b211f-9838-4736-8efe-978e8ad4da6d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:10.168428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 97/203] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:10 2022] GET /identity/v3/role_assignments?user.id=eaa273ae467c455db58a3326415ce7ea&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=e9245ca6f01f444dbbd7fe5e6121449c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:10.168428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b211f-9838-4736-8efe-978e8ad4da6d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:10.223885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee06a48-c27c-48ca-9515-ba61e3364e7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:10.225860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee06a48-c27c-48ca-9515-ba61e3364e7f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:10.275007 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b211f-9838-4736-8efe-978e8ad4da6d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:10.292786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:10.325398 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e9e28-b84c-4758-a3df-ebb01d8fbf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:10.326321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e9e28-b84c-4758-a3df-ebb01d8fbf38 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:10.326703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e9e28-b84c-4758-a3df-ebb01d8fbf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:10.326978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e9e28-b84c-4758-a3df-ebb01d8fbf38 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:10.329858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e9e28-b84c-4758-a3df-ebb01d8fbf38 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:10.331459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e9e28-b84c-4758-a3df-ebb01d8fbf38 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:10.333438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 98/204] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:10 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:10.341497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e9e28-b84c-4758-a3df-ebb01d8fbf38 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:10.401164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 107/205] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:10.408083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee06a48-c27c-48ca-9515-ba61e3364e7f None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:10.459784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e9e28-b84c-4758-a3df-ebb01d8fbf38 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:10.483643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:10.501374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6bfd9-32a4-46a6-b5d1-5c5ee06891a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:10.502047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6bfd9-32a4-46a6-b5d1-5c5ee06891a9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:10.502330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6bfd9-32a4-46a6-b5d1-5c5ee06891a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:10.502606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6bfd9-32a4-46a6-b5d1-5c5ee06891a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:10.503534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6bfd9-32a4-46a6-b5d1-5c5ee06891a9 None admin] RBAC: Adding query filter params (role.id=c45c9440d2b8447ab50b814c6be35bf1, scope.project.id=defbc73a230d43b5afdb17c929c4e61a, user.id=70c3a8b8bbd04186bf178525ae8fb4af) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:10.503848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6bfd9-32a4-46a6-b5d1-5c5ee06891a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:10.505502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6bfd9-32a4-46a6-b5d1-5c5ee06891a9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:10.512073 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 99/206] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:10 2022] GET /identity/v3/role_assignments?user.id=70c3a8b8bbd04186bf178525ae8fb4af&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=c45c9440d2b8447ab50b814c6be35bf1 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:10.553916 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee06a48-c27c-48ca-9515-ba61e3364e7f None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:10.596211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149898d-bdfa-4916-b3ab-b1d324bd7063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:10.597503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149898d-bdfa-4916-b3ab-b1d324bd7063 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:10.598006 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149898d-bdfa-4916-b3ab-b1d324bd7063 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:10.598471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149898d-bdfa-4916-b3ab-b1d324bd7063 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:10.599934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149898d-bdfa-4916-b3ab-b1d324bd7063 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:10.601700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149898d-bdfa-4916-b3ab-b1d324bd7063 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:10.604527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 108/207] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:10 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:10.610088 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6bfd9-32a4-46a6-b5d1-5c5ee06891a9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:10.676352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:10.756210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6bfd9-32a4-46a6-b5d1-5c5ee06891a9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:10.797192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa9fbb-96e8-400a-bff4-c972e4a4cf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:10.797862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa9fbb-96e8-400a-bff4-c972e4a4cf51 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:10.798148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa9fbb-96e8-400a-bff4-c972e4a4cf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:10.798425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa9fbb-96e8-400a-bff4-c972e4a4cf51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:10.799310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa9fbb-96e8-400a-bff4-c972e4a4cf51 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=defbc73a230d43b5afdb17c929c4e61a, user.id=70c3a8b8bbd04186bf178525ae8fb4af) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:10.799623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa9fbb-96e8-400a-bff4-c972e4a4cf51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:10.801135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa9fbb-96e8-400a-bff4-c972e4a4cf51 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:10.807746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 100/208] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:10 2022] GET /identity/v3/role_assignments?user.id=70c3a8b8bbd04186bf178525ae8fb4af&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:11.465467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5337-2691-48f7-b656-35a57a7e072b None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:11.466327 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5337-2691-48f7-b656-35a57a7e072b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:11.466754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5337-2691-48f7-b656-35a57a7e072b None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:11.468619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 109/209] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:11.474277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf777e-90d9-461e-8ec7-bf0cf25bfbd9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:11.474818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf777e-90d9-461e-8ec7-bf0cf25bfbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:11.475314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf777e-90d9-461e-8ec7-bf0cf25bfbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:11.492246 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-06bf777e-90d9-461e-8ec7-bf0cf25bfbd9 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:11.493793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-06bf777e-90d9-461e-8ec7-bf0cf25bfbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:11.548309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeb189-5363-44f7-be2c-c5e8af620a59 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:11.548978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeb189-5363-44f7-be2c-c5e8af620a59 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:11.549462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeb189-5363-44f7-be2c-c5e8af620a59 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:11.551026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 110/210] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:51:11.556337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686c015-aa93-4709-b231-ecd9aea529a0 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:11.557463 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686c015-aa93-4709-b231-ecd9aea529a0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:11.558135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686c015-aa93-4709-b231-ecd9aea529a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:11.567534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 101/211] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:11.576005 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-1686c015-aa93-4709-b231-ecd9aea529a0 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:11.577458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1686c015-aa93-4709-b231-ecd9aea529a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:11.592325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf777e-90d9-461e-8ec7-bf0cf25bfbd9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:11.646401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:11.657972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 111/212] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:11.688277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686c015-aa93-4709-b231-ecd9aea529a0 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:11.730158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf777e-90d9-461e-8ec7-bf0cf25bfbd9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:11.740171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:11.777543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7109216-0b5a-4a2e-a87c-5411c6aed4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:11.778508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7109216-0b5a-4a2e-a87c-5411c6aed4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:11.778923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7109216-0b5a-4a2e-a87c-5411c6aed4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:11.779213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7109216-0b5a-4a2e-a87c-5411c6aed4d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:11.792856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7109216-0b5a-4a2e-a87c-5411c6aed4d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:11.801523 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686c015-aa93-4709-b231-ecd9aea529a0 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d7109216-0b5a-4a2e-a87c-5411c6aed4d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:11.802388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:11.803252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:11.804046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 102/213] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:11.811261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7109216-0b5a-4a2e-a87c-5411c6aed4d2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:11.847758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255367be-7f74-49de-a93c-a9dd5cff1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:11.848606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255367be-7f74-49de-a93c-a9dd5cff1c34 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:11.848895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255367be-7f74-49de-a93c-a9dd5cff1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:11.849174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255367be-7f74-49de-a93c-a9dd5cff1c34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:11.850394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255367be-7f74-49de-a93c-a9dd5cff1c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:11.852858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255367be-7f74-49de-a93c-a9dd5cff1c34 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:11.855484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 112/214] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:11.861012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255367be-7f74-49de-a93c-a9dd5cff1c34 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:11.932557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:11.938423 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7109216-0b5a-4a2e-a87c-5411c6aed4d2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:11.988090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8fcdc-6b42-4404-97c0-75a222701fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:11.988090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8fcdc-6b42-4404-97c0-75a222701fcc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:11.988090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8fcdc-6b42-4404-97c0-75a222701fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:11.988090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8fcdc-6b42-4404-97c0-75a222701fcc None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:11.988450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8fcdc-6b42-4404-97c0-75a222701fcc None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:11.988543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255367be-7f74-49de-a93c-a9dd5cff1c34 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:11.988618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8fcdc-6b42-4404-97c0-75a222701fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:11.990153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8fcdc-6b42-4404-97c0-75a222701fcc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:12.000960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 103/215] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:12.004731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:12.013597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8fcdc-6b42-4404-97c0-75a222701fcc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:12.026847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf60088-33db-4289-9589-03370ddd4ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:12.027514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf60088-33db-4289-9589-03370ddd4ea3 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:12.027794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf60088-33db-4289-9589-03370ddd4ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:12.028058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf60088-33db-4289-9589-03370ddd4ea3 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:12.028865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf60088-33db-4289-9589-03370ddd4ea3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:12.030363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf60088-33db-4289-9589-03370ddd4ea3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:12.054804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 113/216] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:51:11 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:51:12.139700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8fcdc-6b42-4404-97c0-75a222701fcc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:12.187460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b627dd-44d9-447f-8ffe-13e7a09ec4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:12.188797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b627dd-44d9-447f-8ffe-13e7a09ec4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:12.189377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b627dd-44d9-447f-8ffe-13e7a09ec4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:12.189929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b627dd-44d9-447f-8ffe-13e7a09ec4a1 None admin] PATH_INFO: `/v3/users/4c1388585e3345cf993c6d5a3aa08b7f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:12.206673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b627dd-44d9-447f-8ffe-13e7a09ec4a1 None admin] RBAC: Authorizing `identity:get_user(user_id=4c1388585e3345cf993c6d5a3aa08b7f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:12.208714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b627dd-44d9-447f-8ffe-13e7a09ec4a1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:12.210781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 104/217] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:12 2022] GET /identity/v3/users/4c1388585e3345cf993c6d5a3aa08b7f => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:12.217497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf60088-33db-4289-9589-03370ddd4ea3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:12.327380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf60088-33db-4289-9589-03370ddd4ea3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:12.362284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f8fc2-a19e-4e74-95ee-3eef92438dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:12.362947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f8fc2-a19e-4e74-95ee-3eef92438dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:12.363229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f8fc2-a19e-4e74-95ee-3eef92438dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:12.363505 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f8fc2-a19e-4e74-95ee-3eef92438dc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:12.405563 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:12.445876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f8fc2-a19e-4e74-95ee-3eef92438dc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:12.447366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f8fc2-a19e-4e74-95ee-3eef92438dc9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:12.563421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 114/218] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:12.569038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b627dd-44d9-447f-8ffe-13e7a09ec4a1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:12.590647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:12.680728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b627dd-44d9-447f-8ffe-13e7a09ec4a1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:12.716959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21412db7-082d-4c2d-b576-4b6dce4d78c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:12.717838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21412db7-082d-4c2d-b576-4b6dce4d78c6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:12.718121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21412db7-082d-4c2d-b576-4b6dce4d78c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:12.718399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21412db7-082d-4c2d-b576-4b6dce4d78c6 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:12.719601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21412db7-082d-4c2d-b576-4b6dce4d78c6 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:12.721118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21412db7-082d-4c2d-b576-4b6dce4d78c6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:12.723126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 105/219] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:12 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:12.729548 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f8fc2-a19e-4e74-95ee-3eef92438dc9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:12.810141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:12.842833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f8fc2-a19e-4e74-95ee-3eef92438dc9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:12.878359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca88c2-e54b-4c08-9916-02cc7a9ce3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:12.879030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca88c2-e54b-4c08-9916-02cc7a9ce3fc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:12.879314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca88c2-e54b-4c08-9916-02cc7a9ce3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:12.879592 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca88c2-e54b-4c08-9916-02cc7a9ce3fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:12.880460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca88c2-e54b-4c08-9916-02cc7a9ce3fc None admin] RBAC: Adding query filter params (role.id=61490cf1d7334421b84ff67552b79bfc, scope.project.id=defbc73a230d43b5afdb17c929c4e61a, user.id=4c1388585e3345cf993c6d5a3aa08b7f) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:12.880780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca88c2-e54b-4c08-9916-02cc7a9ce3fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:12.882261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca88c2-e54b-4c08-9916-02cc7a9ce3fc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:12.889643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c6bbd-a400-4066-8b6d-5ccd864fc274 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:12.890110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c6bbd-a400-4066-8b6d-5ccd864fc274 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:12.890293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c6bbd-a400-4066-8b6d-5ccd864fc274 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:12.891594 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 106/220] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:12.892935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 115/221] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:12 2022] GET /identity/v3/role_assignments?user.id=4c1388585e3345cf993c6d5a3aa08b7f&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=61490cf1d7334421b84ff67552b79bfc => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:12.914540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcec87-074a-4c30-816e-c26ea8fa5315 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:12.914837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcec87-074a-4c30-816e-c26ea8fa5315 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:12.915122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcec87-074a-4c30-816e-c26ea8fa5315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:12.939058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-aafcec87-074a-4c30-816e-c26ea8fa5315 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:12.940708 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-aafcec87-074a-4c30-816e-c26ea8fa5315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:13.049186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 107/222] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:13.085964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca88c2-e54b-4c08-9916-02cc7a9ce3fc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:13.135459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f975f-50be-4a69-a28c-003e7eaebd36 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:13.136055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f975f-50be-4a69-a28c-003e7eaebd36 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:13.136572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f975f-50be-4a69-a28c-003e7eaebd36 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:13.138393 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 108/223] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:51:13.143909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b5027-07fe-4b15-9e53-0d238a7e2247 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:13.144481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b5027-07fe-4b15-9e53-0d238a7e2247 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:13.144959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b5027-07fe-4b15-9e53-0d238a7e2247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:13.173806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-044b5027-07fe-4b15-9e53-0d238a7e2247 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:13.175026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-044b5027-07fe-4b15-9e53-0d238a7e2247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:13.189582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:13.207706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca88c2-e54b-4c08-9916-02cc7a9ce3fc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:13.250882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32e818-2b1f-4c63-aa0f-3be4603d5396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:13.251706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32e818-2b1f-4c63-aa0f-3be4603d5396 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:13.251989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32e818-2b1f-4c63-aa0f-3be4603d5396 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:13.252263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32e818-2b1f-4c63-aa0f-3be4603d5396 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:13.265225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32e818-2b1f-4c63-aa0f-3be4603d5396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:13.272715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-6b32e818-2b1f-4c63-aa0f-3be4603d5396 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:13.278090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:13.280341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:13.281152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 116/224] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:13.281792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 109/225] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:13.285162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32e818-2b1f-4c63-aa0f-3be4603d5396 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:13.325479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b5027-07fe-4b15-9e53-0d238a7e2247 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:13.399375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32e818-2b1f-4c63-aa0f-3be4603d5396 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:13.437045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394da6cc-1d09-44f7-ab1f-fe04afd9e5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:13.437733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394da6cc-1d09-44f7-ab1f-fe04afd9e5cc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:13.438026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394da6cc-1d09-44f7-ab1f-fe04afd9e5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:13.438311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394da6cc-1d09-44f7-ab1f-fe04afd9e5cc None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:13.439252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394da6cc-1d09-44f7-ab1f-fe04afd9e5cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:13.439578 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394da6cc-1d09-44f7-ab1f-fe04afd9e5cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:13.441257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394da6cc-1d09-44f7-ab1f-fe04afd9e5cc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:13.453367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 117/226] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:13.460164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394da6cc-1d09-44f7-ab1f-fe04afd9e5cc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:13.548758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:13.572055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394da6cc-1d09-44f7-ab1f-fe04afd9e5cc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:13.609530 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3986cd7-f849-401f-8893-9d5ec8b323c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:13.610532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986cd7-f849-401f-8893-9d5ec8b323c7 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:13.610892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986cd7-f849-401f-8893-9d5ec8b323c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:13.611243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986cd7-f849-401f-8893-9d5ec8b323c7 None admin] PATH_INFO: `/v3/users/70c3a8b8bbd04186bf178525ae8fb4af` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:13.612543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3986cd7-f849-401f-8893-9d5ec8b323c7 None admin] RBAC: Authorizing `identity:get_user(user_id=70c3a8b8bbd04186bf178525ae8fb4af)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:13.614081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3986cd7-f849-401f-8893-9d5ec8b323c7 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:13.616361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 118/227] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:13 2022] GET /identity/v3/users/70c3a8b8bbd04186bf178525ae8fb4af => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:13.625526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3986cd7-f849-401f-8893-9d5ec8b323c7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:13.638564 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b5027-07fe-4b15-9e53-0d238a7e2247 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:13.710984 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3849fc4-4544-460f-98c2-9d2822d388d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:13.711731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3849fc4-4544-460f-98c2-9d2822d388d0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:13.712014 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3849fc4-4544-460f-98c2-9d2822d388d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:13.712289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3849fc4-4544-460f-98c2-9d2822d388d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:13.733725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3849fc4-4544-460f-98c2-9d2822d388d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:13.744832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3986cd7-f849-401f-8893-9d5ec8b323c7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d3849fc4-4544-460f-98c2-9d2822d388d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:13.758212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:13.760380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:13.762272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 110/228] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:51:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:13.767995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3849fc4-4544-460f-98c2-9d2822d388d0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:13.772562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:13.776732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:13.798016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512a012-b32a-468c-bd7a-4ef96894b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:13.800270 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512a012-b32a-468c-bd7a-4ef96894b8df None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:13.804083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512a012-b32a-468c-bd7a-4ef96894b8df None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:13.804083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512a012-b32a-468c-bd7a-4ef96894b8df None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:13.804912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512a012-b32a-468c-bd7a-4ef96894b8df None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:13.806373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512a012-b32a-468c-bd7a-4ef96894b8df None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:13.809723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 119/229] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:13 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:13.816494 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512a012-b32a-468c-bd7a-4ef96894b8df None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:13.942893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3849fc4-4544-460f-98c2-9d2822d388d0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:14.000122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca1ac0-163c-4344-894d-42ddaab32ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:14.000836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca1ac0-163c-4344-894d-42ddaab32ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:14.001119 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca1ac0-163c-4344-894d-42ddaab32ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:14.001392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca1ac0-163c-4344-894d-42ddaab32ce3 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:14.002146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca1ac0-163c-4344-894d-42ddaab32ce3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:14.002452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca1ac0-163c-4344-894d-42ddaab32ce3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:14.003861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca1ac0-163c-4344-894d-42ddaab32ce3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:14.021730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 111/230] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:14.029291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca1ac0-163c-4344-894d-42ddaab32ce3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:14.035572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512a012-b32a-468c-bd7a-4ef96894b8df None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:14.104424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de4b6d-9776-4d1a-a8b5-64e1847969dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:14.105343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de4b6d-9776-4d1a-a8b5-64e1847969dd None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:14.105625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de4b6d-9776-4d1a-a8b5-64e1847969dd None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:14.105904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de4b6d-9776-4d1a-a8b5-64e1847969dd None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a/users/70c3a8b8bbd04186bf178525ae8fb4af/roles/c45c9440d2b8447ab50b814c6be35bf1` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:14.109319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de4b6d-9776-4d1a-a8b5-64e1847969dd None admin] RBAC: Authorizing `identity:create_grant(project_id=defbc73a230d43b5afdb17c929c4e61a, user_id=70c3a8b8bbd04186bf178525ae8fb4af, role_id=c45c9440d2b8447ab50b814c6be35bf1)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:14.110974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de4b6d-9776-4d1a-a8b5-64e1847969dd None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:14.132282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 120/231] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:51:13 2022] PUT /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a/users/70c3a8b8bbd04186bf178525ae8fb4af/roles/c45c9440d2b8447ab50b814c6be35bf1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:14.151096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:14.189994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca1ac0-163c-4344-894d-42ddaab32ce3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:14.237613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b999d6c-1ed0-44cc-86df-625cf77d0197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:14.238505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999d6c-1ed0-44cc-86df-625cf77d0197 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:14.238786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999d6c-1ed0-44cc-86df-625cf77d0197 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:14.239061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999d6c-1ed0-44cc-86df-625cf77d0197 None admin] PATH_INFO: `/v3/users/70c3a8b8bbd04186bf178525ae8fb4af` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:14.240327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b999d6c-1ed0-44cc-86df-625cf77d0197 None admin] RBAC: Authorizing `identity:get_user(user_id=70c3a8b8bbd04186bf178525ae8fb4af)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:14.241832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b999d6c-1ed0-44cc-86df-625cf77d0197 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:14.243849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 112/232] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:14 2022] GET /identity/v3/users/70c3a8b8bbd04186bf178525ae8fb4af => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:14.249578 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de4b6d-9776-4d1a-a8b5-64e1847969dd None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:14.340175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:14.414434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de4b6d-9776-4d1a-a8b5-64e1847969dd None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:14.457404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08440a-0fb7-475d-bd73-35840cdcd0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:14.458285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08440a-0fb7-475d-bd73-35840cdcd0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:14.458567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08440a-0fb7-475d-bd73-35840cdcd0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:14.458845 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08440a-0fb7-475d-bd73-35840cdcd0a2 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:14.460051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08440a-0fb7-475d-bd73-35840cdcd0a2 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:14.461567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08440a-0fb7-475d-bd73-35840cdcd0a2 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:14.464057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 121/233] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:14 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:14.469682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b999d6c-1ed0-44cc-86df-625cf77d0197 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:14.595125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b999d6c-1ed0-44cc-86df-625cf77d0197 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:14.655775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fdca5-d9d2-40a4-a1e2-7d14368edb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:14.656722 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fdca5-d9d2-40a4-a1e2-7d14368edb21 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:14.657055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fdca5-d9d2-40a4-a1e2-7d14368edb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:14.657339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fdca5-d9d2-40a4-a1e2-7d14368edb21 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a/users/70c3a8b8bbd04186bf178525ae8fb4af/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:14.662039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fdca5-d9d2-40a4-a1e2-7d14368edb21 None admin] RBAC: Authorizing `identity:create_grant(project_id=defbc73a230d43b5afdb17c929c4e61a, user_id=70c3a8b8bbd04186bf178525ae8fb4af, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:14.663538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1fdca5-d9d2-40a4-a1e2-7d14368edb21 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:14.667554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:14.684929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 113/234] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:51:14 2022] PUT /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a/users/70c3a8b8bbd04186bf178525ae8fb4af/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:15.082678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d82c34-c132-454a-9871-a93ae5cda47f None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:15.083177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d82c34-c132-454a-9871-a93ae5cda47f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:15.083280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d82c34-c132-454a-9871-a93ae5cda47f None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:15.084918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 122/235] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:15.090445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43be4f6-bb51-47e3-9522-b02ce381c56c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:15.090852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43be4f6-bb51-47e3-9522-b02ce381c56c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:15.091038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43be4f6-bb51-47e3-9522-b02ce381c56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:15.111194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f43be4f6-bb51-47e3-9522-b02ce381c56c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:15.112423 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f43be4f6-bb51-47e3-9522-b02ce381c56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:15.186326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 114/236] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:15.213276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d82c34-c132-454a-9871-a93ae5cda47f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:15.342836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d82c34-c132-454a-9871-a93ae5cda47f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:15.363198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:15.378709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9122e-6927-4bba-9335-8cc513d2ac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:15.379517 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9122e-6927-4bba-9335-8cc513d2ac65 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:15.379801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9122e-6927-4bba-9335-8cc513d2ac65 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:15.380075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9122e-6927-4bba-9335-8cc513d2ac65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:15.381279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9122e-6927-4bba-9335-8cc513d2ac65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:15.382804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9122e-6927-4bba-9335-8cc513d2ac65 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:15.384966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 123/237] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:15.390295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43be4f6-bb51-47e3-9522-b02ce381c56c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:15.395330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90cdd6-e668-4045-ba45-55d6567066d9 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:15.395623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90cdd6-e668-4045-ba45-55d6567066d9 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:15.395902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90cdd6-e668-4045-ba45-55d6567066d9 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:15.397430 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 124/238] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:51:15.403543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d3602-04a7-4d51-bd7a-48d51824747c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:15.403831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d3602-04a7-4d51-bd7a-48d51824747c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:15.404112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d3602-04a7-4d51-bd7a-48d51824747c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:15.422238 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e27d3602-04a7-4d51-bd7a-48d51824747c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:15.423515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e27d3602-04a7-4d51-bd7a-48d51824747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:15.537571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43be4f6-bb51-47e3-9522-b02ce381c56c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:15.539108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 125/239] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:15.562890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d3602-04a7-4d51-bd7a-48d51824747c None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:15.594924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:15.615143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fae6e-ed68-4e9b-9e5a-71c5d10f6f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:15.615780 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fae6e-ed68-4e9b-9e5a-71c5d10f6f14 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:15.616070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fae6e-ed68-4e9b-9e5a-71c5d10f6f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:15.616352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fae6e-ed68-4e9b-9e5a-71c5d10f6f14 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:15.617106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069fae6e-ed68-4e9b-9e5a-71c5d10f6f14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:15.618573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069fae6e-ed68-4e9b-9e5a-71c5d10f6f14 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:15.657526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 115/240] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:51:15 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:51:15.670456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f7055-48c9-4dd6-89c3-fb2e377083ac None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:15.670921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f7055-48c9-4dd6-89c3-fb2e377083ac None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:15.671497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f7055-48c9-4dd6-89c3-fb2e377083ac None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:15.673303 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 116/241] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:51:15.682884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9c973-08c1-4987-882c-96b1e7c83878 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:15.683198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9c973-08c1-4987-882c-96b1e7c83878 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:15.683448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9c973-08c1-4987-882c-96b1e7c83878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:15.717299 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c8f9c973-08c1-4987-882c-96b1e7c83878 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:15.719576 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c8f9c973-08c1-4987-882c-96b1e7c83878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:15.725334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d3602-04a7-4d51-bd7a-48d51824747c None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:15.763186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:15.799858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f14716-2440-4b70-9e2f-a76c913921f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:15.800631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f14716-2440-4b70-9e2f-a76c913921f8 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:15.800919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f14716-2440-4b70-9e2f-a76c913921f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:15.801200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f14716-2440-4b70-9e2f-a76c913921f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:15.825576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f14716-2440-4b70-9e2f-a76c913921f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-f9f14716-2440-4b70-9e2f-a76c913921f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:15.833507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:15.834452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:15.835232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 126/242] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:15.846248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f14716-2440-4b70-9e2f-a76c913921f8 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:15.862595 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 117/243] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:15.886555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9c973-08c1-4987-882c-96b1e7c83878 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:15.999237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9c973-08c1-4987-882c-96b1e7c83878 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:16.005814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f14716-2440-4b70-9e2f-a76c913921f8 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:16.014292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:16.040652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409544b-59a0-4057-b7c6-111f995c7a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:16.041983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409544b-59a0-4057-b7c6-111f995c7a4a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:16.042583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409544b-59a0-4057-b7c6-111f995c7a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:16.043084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409544b-59a0-4057-b7c6-111f995c7a4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:16.045779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdf2cb-64ff-4208-8b8e-b276b2d720e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:16.046400 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdf2cb-64ff-4208-8b8e-b276b2d720e2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:16.046682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdf2cb-64ff-4208-8b8e-b276b2d720e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:16.046956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdf2cb-64ff-4208-8b8e-b276b2d720e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:16.047738 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdf2cb-64ff-4208-8b8e-b276b2d720e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:16.048014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdf2cb-64ff-4208-8b8e-b276b2d720e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:16.049437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdf2cb-64ff-4208-8b8e-b276b2d720e2 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:16.057177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409544b-59a0-4057-b7c6-111f995c7a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:16.059471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 127/244] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:16.066617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdf2cb-64ff-4208-8b8e-b276b2d720e2 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-3409544b-59a0-4057-b7c6-111f995c7a4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:16.068960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:16.069831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:16.070642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:16.074264 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 118/245] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:16.081686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409544b-59a0-4057-b7c6-111f995c7a4a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:16.119927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:16.201227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409544b-59a0-4057-b7c6-111f995c7a4a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:16.207919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdf2cb-64ff-4208-8b8e-b276b2d720e2 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:16.250660 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6ff0f-1463-4910-993a-4c4dc5b335e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:16.251231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a718e6b-1ea6-455c-8b8f-0474474daf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:16.251977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6ff0f-1463-4910-993a-4c4dc5b335e7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:16.252151 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a718e6b-1ea6-455c-8b8f-0474474daf61 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:16.252380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a718e6b-1ea6-455c-8b8f-0474474daf61 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:16.252684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a718e6b-1ea6-455c-8b8f-0474474daf61 None admin] PATH_INFO: `/v3/users/a75a6f58e11a4643a6539f43589c90a7` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:16.253039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6ff0f-1463-4910-993a-4c4dc5b335e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:16.253581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6ff0f-1463-4910-993a-4c4dc5b335e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:16.254525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6ff0f-1463-4910-993a-4c4dc5b335e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:16.255021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6ff0f-1463-4910-993a-4c4dc5b335e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:16.256628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6ff0f-1463-4910-993a-4c4dc5b335e7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:16.266939 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 119/246] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:16.272862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6ff0f-1463-4910-993a-4c4dc5b335e7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:16.274439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a718e6b-1ea6-455c-8b8f-0474474daf61 None admin] RBAC: Authorizing `identity:get_user(user_id=a75a6f58e11a4643a6539f43589c90a7)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:16.278498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a718e6b-1ea6-455c-8b8f-0474474daf61 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:16.288502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 128/247] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:16 2022] GET /identity/v3/users/a75a6f58e11a4643a6539f43589c90a7 => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:16.288502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a718e6b-1ea6-455c-8b8f-0474474daf61 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:16.320204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:16.416274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:16.420833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a718e6b-1ea6-455c-8b8f-0474474daf61 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:16.463856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6ff0f-1463-4910-993a-4c4dc5b335e7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:16.470190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744d53a-0821-4b93-a1a3-603a5fc4c560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:16.471036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744d53a-0821-4b93-a1a3-603a5fc4c560 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:16.471557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744d53a-0821-4b93-a1a3-603a5fc4c560 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:16.471979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744d53a-0821-4b93-a1a3-603a5fc4c560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:16.552431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744d53a-0821-4b93-a1a3-603a5fc4c560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:16.554260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744d53a-0821-4b93-a1a3-603a5fc4c560 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:16.556889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed02d9-6225-45e8-a06d-dfb448151b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:16.558286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed02d9-6225-45e8-a06d-dfb448151b26 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:16.558286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed02d9-6225-45e8-a06d-dfb448151b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:16.558636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed02d9-6225-45e8-a06d-dfb448151b26 None admin] PATH_INFO: `/v3/users/4c1388585e3345cf993c6d5a3aa08b7f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:16.564074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed02d9-6225-45e8-a06d-dfb448151b26 None admin] RBAC: Authorizing `identity:get_user(user_id=4c1388585e3345cf993c6d5a3aa08b7f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:16.565655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed02d9-6225-45e8-a06d-dfb448151b26 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:16.567526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 120/248] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:16 2022] GET /identity/v3/users/4c1388585e3345cf993c6d5a3aa08b7f => generated 297 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:16.577498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed02d9-6225-45e8-a06d-dfb448151b26 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:16.663282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 129/249] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:16.668809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:16.670755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744d53a-0821-4b93-a1a3-603a5fc4c560 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:16.811460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed02d9-6225-45e8-a06d-dfb448151b26 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:16.828876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:16.860648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744d53a-0821-4b93-a1a3-603a5fc4c560 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:16.894064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217e1e5-d974-4577-8396-c04614aeb125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:16.894967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217e1e5-d974-4577-8396-c04614aeb125 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:16.895253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217e1e5-d974-4577-8396-c04614aeb125 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:16.895535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217e1e5-d974-4577-8396-c04614aeb125 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:16.896763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217e1e5-d974-4577-8396-c04614aeb125 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:16.898240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217e1e5-d974-4577-8396-c04614aeb125 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:16.900020 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 121/250] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:16 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:16.918784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217e1e5-d974-4577-8396-c04614aeb125 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:16.926514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7165144-531a-43cc-98c8-152c56ab3a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:16.927685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7165144-531a-43cc-98c8-152c56ab3a88 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:16.928143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7165144-531a-43cc-98c8-152c56ab3a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:16.928619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7165144-531a-43cc-98c8-152c56ab3a88 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:16.933546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7165144-531a-43cc-98c8-152c56ab3a88 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:16.935230 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7165144-531a-43cc-98c8-152c56ab3a88 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:16.937176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 130/251] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:16 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:16.942863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7165144-531a-43cc-98c8-152c56ab3a88 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:17.041184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:17.078789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7165144-531a-43cc-98c8-152c56ab3a88 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:17.116560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e078440-e016-464b-a01a-bb0219190f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:17.117495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e078440-e016-464b-a01a-bb0219190f01 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:17.118190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e078440-e016-464b-a01a-bb0219190f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:17.118637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e078440-e016-464b-a01a-bb0219190f01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:17.120192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e078440-e016-464b-a01a-bb0219190f01 None admin] RBAC: Adding query filter params (role.id=e9245ca6f01f444dbbd7fe5e6121449c, scope.project.id=defbc73a230d43b5afdb17c929c4e61a, user.id=a75a6f58e11a4643a6539f43589c90a7) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:17.121133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e078440-e016-464b-a01a-bb0219190f01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:17.122820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e078440-e016-464b-a01a-bb0219190f01 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:17.129463 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 131/252] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:16 2022] GET /identity/v3/role_assignments?user.id=a75a6f58e11a4643a6539f43589c90a7&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=e9245ca6f01f444dbbd7fe5e6121449c => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:17.171411 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217e1e5-d974-4577-8396-c04614aeb125 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:17.203508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3298518-817e-4baf-b1ca-a19abfa92378 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:17.205208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3298518-817e-4baf-b1ca-a19abfa92378 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:17.205923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3298518-817e-4baf-b1ca-a19abfa92378 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:17.209215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 132/253] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:17.215316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ee8c7-a69a-41bb-a7ad-510936a5a330 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:17.215881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ee8c7-a69a-41bb-a7ad-510936a5a330 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:17.216332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ee8c7-a69a-41bb-a7ad-510936a5a330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:17.250434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-5e7ee8c7-a69a-41bb-a7ad-510936a5a330 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:17.252465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5e7ee8c7-a69a-41bb-a7ad-510936a5a330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:17.256938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de3b09-9ac0-4c99-a1d2-478c0fa93492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:17.258484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3b09-9ac0-4c99-a1d2-478c0fa93492 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:17.258941 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3b09-9ac0-4c99-a1d2-478c0fa93492 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:17.259387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de3b09-9ac0-4c99-a1d2-478c0fa93492 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a/users/4c1388585e3345cf993c6d5a3aa08b7f/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:17.269665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de3b09-9ac0-4c99-a1d2-478c0fa93492 None admin] RBAC: Authorizing `identity:create_grant(project_id=defbc73a230d43b5afdb17c929c4e61a, user_id=4c1388585e3345cf993c6d5a3aa08b7f, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:17.271439 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de3b09-9ac0-4c99-a1d2-478c0fa93492 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:17.303264 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 122/254] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:51:16 2022] PUT /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a/users/4c1388585e3345cf993c6d5a3aa08b7f/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:17.367116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 133/255] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:17.414801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de3b09-9ac0-4c99-a1d2-478c0fa93492 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:17.466918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:17.558985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de3b09-9ac0-4c99-a1d2-478c0fa93492 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:17.595962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84c501-9283-46e4-a451-2b4030ca1f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:17.596831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84c501-9283-46e4-a451-2b4030ca1f53 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:17.597118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84c501-9283-46e4-a451-2b4030ca1f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:17.597397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84c501-9283-46e4-a451-2b4030ca1f53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:17.609657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84c501-9283-46e4-a451-2b4030ca1f53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-8b84c501-9283-46e4-a451-2b4030ca1f53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:17.619892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:17.621187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:17.621981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 123/256] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:17.626670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ee8c7-a69a-41bb-a7ad-510936a5a330 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:17.706026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:17.759724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ee8c7-a69a-41bb-a7ad-510936a5a330 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:17.798770 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4e72e-7293-4efc-8f63-3f2ecf0d1283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:17.799473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e72e-7293-4efc-8f63-3f2ecf0d1283 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:17.799758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e72e-7293-4efc-8f63-3f2ecf0d1283 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:17.800045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e72e-7293-4efc-8f63-3f2ecf0d1283 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:17.800856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4e72e-7293-4efc-8f63-3f2ecf0d1283 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:17.801169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4e72e-7293-4efc-8f63-3f2ecf0d1283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:17.802630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4e72e-7293-4efc-8f63-3f2ecf0d1283 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:17.813682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 134/257] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:17.819487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84c501-9283-46e4-a451-2b4030ca1f53 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:17.952021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:17.967249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84c501-9283-46e4-a451-2b4030ca1f53 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:18.022811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc269bdb-fb71-4a09-b585-091b2731d2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:18.023717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc269bdb-fb71-4a09-b585-091b2731d2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:18.024003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc269bdb-fb71-4a09-b585-091b2731d2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:18.024281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc269bdb-fb71-4a09-b585-091b2731d2e5 None admin] PATH_INFO: `/v3/users/70c3a8b8bbd04186bf178525ae8fb4af` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:18.025715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc269bdb-fb71-4a09-b585-091b2731d2e5 None admin] RBAC: Authorizing `identity:get_user(user_id=70c3a8b8bbd04186bf178525ae8fb4af)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:18.027214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc269bdb-fb71-4a09-b585-091b2731d2e5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:18.028963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 124/258] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:17 2022] GET /identity/v3/users/70c3a8b8bbd04186bf178525ae8fb4af => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:18.034500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4e72e-7293-4efc-8f63-3f2ecf0d1283 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:18.148809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4e72e-7293-4efc-8f63-3f2ecf0d1283 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:18.173468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:18.186219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d8979-dcea-4e6f-955b-b8ecfccb99f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:18.187227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d8979-dcea-4e6f-955b-b8ecfccb99f3 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:18.187894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d8979-dcea-4e6f-955b-b8ecfccb99f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:18.188177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d8979-dcea-4e6f-955b-b8ecfccb99f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:18.258807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d8979-dcea-4e6f-955b-b8ecfccb99f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:18.260413 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d8979-dcea-4e6f-955b-b8ecfccb99f3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:18.362090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 135/259] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:18.369997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc269bdb-fb71-4a09-b585-091b2731d2e5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:18.505483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc269bdb-fb71-4a09-b585-091b2731d2e5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:18.551704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee7fcf-e4c6-42e8-a6a4-145439379bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:18.552985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee7fcf-e4c6-42e8-a6a4-145439379bbf None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:18.553470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee7fcf-e4c6-42e8-a6a4-145439379bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:18.553991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee7fcf-e4c6-42e8-a6a4-145439379bbf None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:18.557809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee7fcf-e4c6-42e8-a6a4-145439379bbf None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:18.559517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee7fcf-e4c6-42e8-a6a4-145439379bbf None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:18.561475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 125/260] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:18 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:18.567414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d8979-dcea-4e6f-955b-b8ecfccb99f3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:18.597760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:18.706172 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d8979-dcea-4e6f-955b-b8ecfccb99f3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:18.760134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da013b-55e7-4780-94d2-f457734f2fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:18.761177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da013b-55e7-4780-94d2-f457734f2fcb None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:18.761663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da013b-55e7-4780-94d2-f457734f2fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:18.762136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da013b-55e7-4780-94d2-f457734f2fcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:18.763163 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da013b-55e7-4780-94d2-f457734f2fcb None admin] RBAC: Adding query filter params (role.id=c45c9440d2b8447ab50b814c6be35bf1, scope.project.id=defbc73a230d43b5afdb17c929c4e61a, user.id=70c3a8b8bbd04186bf178525ae8fb4af) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:18.763696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da013b-55e7-4780-94d2-f457734f2fcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:18.765377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da013b-55e7-4780-94d2-f457734f2fcb None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:18.777135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 136/261] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:18 2022] GET /identity/v3/role_assignments?user.id=70c3a8b8bbd04186bf178525ae8fb4af&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=c45c9440d2b8447ab50b814c6be35bf1 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:18.827956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a56fc-b559-4543-b8ad-f636a82f3edd None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:18.828663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a56fc-b559-4543-b8ad-f636a82f3edd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:18.829170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a56fc-b559-4543-b8ad-f636a82f3edd None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:18.830858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 126/262] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:18.836927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe756545-da9e-4220-be56-5b22681f81cc None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:18.837497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe756545-da9e-4220-be56-5b22681f81cc None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:18.838208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe756545-da9e-4220-be56-5b22681f81cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:18.860584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-fe756545-da9e-4220-be56-5b22681f81cc None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:18.862119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-fe756545-da9e-4220-be56-5b22681f81cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:18.954055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 137/263] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:18.979681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a56fc-b559-4543-b8ad-f636a82f3edd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:19.020432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:19.115833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a56fc-b559-4543-b8ad-f636a82f3edd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:19.154592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae19a08-7537-4f5e-98cc-cc9c7509ab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:19.155735 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae19a08-7537-4f5e-98cc-cc9c7509ab29 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:19.156217 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae19a08-7537-4f5e-98cc-cc9c7509ab29 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:19.156833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae19a08-7537-4f5e-98cc-cc9c7509ab29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:19.174347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae19a08-7537-4f5e-98cc-cc9c7509ab29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-eae19a08-7537-4f5e-98cc-cc9c7509ab29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:19.183524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:19.184531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:19.185314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:19.188257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 127/264] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:51:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:19.194779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe756545-da9e-4220-be56-5b22681f81cc None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:19.216152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:19.308691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe756545-da9e-4220-be56-5b22681f81cc None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:19.346678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e4ebf-4f9a-432a-98a5-a551a36cd264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:19.347371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e4ebf-4f9a-432a-98a5-a551a36cd264 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:19.347655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e4ebf-4f9a-432a-98a5-a551a36cd264 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:19.347989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e4ebf-4f9a-432a-98a5-a551a36cd264 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:19.348786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e4ebf-4f9a-432a-98a5-a551a36cd264 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:19.349141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e4ebf-4f9a-432a-98a5-a551a36cd264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:19.350539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e4ebf-4f9a-432a-98a5-a551a36cd264 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:19.360050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 138/265] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:19.365383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae19a08-7537-4f5e-98cc-cc9c7509ab29 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:19.463971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:19.490697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae19a08-7537-4f5e-98cc-cc9c7509ab29 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:19.537342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221d62d-6158-48b6-8b81-971674137f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:19.538601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221d62d-6158-48b6-8b81-971674137f4b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:19.539114 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221d62d-6158-48b6-8b81-971674137f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:19.539591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221d62d-6158-48b6-8b81-971674137f4b None admin] PATH_INFO: `/v3/users/70c3a8b8bbd04186bf178525ae8fb4af` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:19.541778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221d62d-6158-48b6-8b81-971674137f4b None admin] RBAC: Authorizing `identity:get_user(user_id=70c3a8b8bbd04186bf178525ae8fb4af)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:19.543481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221d62d-6158-48b6-8b81-971674137f4b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:19.545371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 128/266] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:19 2022] GET /identity/v3/users/70c3a8b8bbd04186bf178525ae8fb4af => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:19.550915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e4ebf-4f9a-432a-98a5-a551a36cd264 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:19.628754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e456ec3-4643-43f3-954c-62ec8ac5ec4c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:19.629359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e456ec3-4643-43f3-954c-62ec8ac5ec4c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:19.629839 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e456ec3-4643-43f3-954c-62ec8ac5ec4c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:19.631970 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 129/267] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:19.639129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768392ee-898d-457c-a135-162cc4913e8e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:19.639712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768392ee-898d-457c-a135-162cc4913e8e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:19.640211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768392ee-898d-457c-a135-162cc4913e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:19.649108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:19.671447 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e4ebf-4f9a-432a-98a5-a551a36cd264 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:19.686184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-768392ee-898d-457c-a135-162cc4913e8e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:19.688207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-768392ee-898d-457c-a135-162cc4913e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:19.719297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457655a0-8a7c-40a2-b371-2d322bd2bf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:19.720309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457655a0-8a7c-40a2-b371-2d322bd2bf0e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:19.720951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457655a0-8a7c-40a2-b371-2d322bd2bf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:19.722239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457655a0-8a7c-40a2-b371-2d322bd2bf0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:19.796543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 130/268] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:19.801861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da4533-8eba-43ea-902b-d611fcaab459 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:19.802167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da4533-8eba-43ea-902b-d611fcaab459 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:19.802399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da4533-8eba-43ea-902b-d611fcaab459 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:19.805236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 131/269] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:19.811095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddca377-46ea-499b-9c43-8945f12e3e57 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:19.811378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddca377-46ea-499b-9c43-8945f12e3e57 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:19.811654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddca377-46ea-499b-9c43-8945f12e3e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:19.821880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457655a0-8a7c-40a2-b371-2d322bd2bf0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:19.823621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457655a0-8a7c-40a2-b371-2d322bd2bf0e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:19.841677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-dddca377-46ea-499b-9c43-8945f12e3e57 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:19.843432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dddca377-46ea-499b-9c43-8945f12e3e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:19.922616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:19.925867 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 132/270] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:19.926964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddca377-46ea-499b-9c43-8945f12e3e57 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:19.943199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:19.953112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 139/271] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:19.961391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457655a0-8a7c-40a2-b371-2d322bd2bf0e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:20.031817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddca377-46ea-499b-9c43-8945f12e3e57 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:20.069749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd8546-db5d-4722-bb6d-ad61834e5007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:20.070581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd8546-db5d-4722-bb6d-ad61834e5007 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.070924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd8546-db5d-4722-bb6d-ad61834e5007 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.071105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd8546-db5d-4722-bb6d-ad61834e5007 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.078043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457655a0-8a7c-40a2-b371-2d322bd2bf0e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:20.083461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd8546-db5d-4722-bb6d-ad61834e5007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-ddfd8546-db5d-4722-bb6d-ad61834e5007 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:20.091997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:20.092954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:20.093749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:20.098198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 133/272] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:20.099225 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd8546-db5d-4722-bb6d-ad61834e5007 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:20.124475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186646f6-0076-498d-bfdb-811d57587a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:20.125757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186646f6-0076-498d-bfdb-811d57587a9a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.126359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186646f6-0076-498d-bfdb-811d57587a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.126859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186646f6-0076-498d-bfdb-811d57587a9a None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.128445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186646f6-0076-498d-bfdb-811d57587a9a None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:20.130246 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186646f6-0076-498d-bfdb-811d57587a9a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:20.132509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 140/273] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:19 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:20.133617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186646f6-0076-498d-bfdb-811d57587a9a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:20.185351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:20.188995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:20.212259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd8546-db5d-4722-bb6d-ad61834e5007 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:20.248707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cd086-4ee0-4611-81e9-c17123158f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:20.249652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd086-4ee0-4611-81e9-c17123158f96 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.250002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd086-4ee0-4611-81e9-c17123158f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.250348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd086-4ee0-4611-81e9-c17123158f96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.251592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cd086-4ee0-4611-81e9-c17123158f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:20.253018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186646f6-0076-498d-bfdb-811d57587a9a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:20.253477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cd086-4ee0-4611-81e9-c17123158f96 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:20.254841 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 134/274] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:20.255692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cd086-4ee0-4611-81e9-c17123158f96 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:20.296604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e0281-1aca-49c2-91a3-3884a554b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:20.297378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e0281-1aca-49c2-91a3-3884a554b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.297776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e0281-1aca-49c2-91a3-3884a554b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.298129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e0281-1aca-49c2-91a3-3884a554b3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.299098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e0281-1aca-49c2-91a3-3884a554b3b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:20.299470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e0281-1aca-49c2-91a3-3884a554b3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:20.301518 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e0281-1aca-49c2-91a3-3884a554b3b3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:20.313639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 141/275] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:20.318256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e0281-1aca-49c2-91a3-3884a554b3b3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:20.404025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cd086-4ee0-4611-81e9-c17123158f96 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:20.449955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2df4a-aea1-468a-be13-2ee03693e75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:20.450552 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2df4a-aea1-468a-be13-2ee03693e75f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.450841 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2df4a-aea1-468a-be13-2ee03693e75f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.451287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2df4a-aea1-468a-be13-2ee03693e75f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.452229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2df4a-aea1-468a-be13-2ee03693e75f None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=defbc73a230d43b5afdb17c929c4e61a, user.id=70c3a8b8bbd04186bf178525ae8fb4af) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:20.452597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2df4a-aea1-468a-be13-2ee03693e75f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:20.454823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2df4a-aea1-468a-be13-2ee03693e75f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:20.460423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e0281-1aca-49c2-91a3-3884a554b3b3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:20.461093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 135/276] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:20 2022] GET /identity/v3/role_assignments?user.id=70c3a8b8bbd04186bf178525ae8fb4af&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:20.461779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2df4a-aea1-468a-be13-2ee03693e75f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:20.472327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:20.473360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:20.502408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39c380-580c-40fc-85d3-02b08635bcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:20.503283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39c380-580c-40fc-85d3-02b08635bcc5 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.503675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39c380-580c-40fc-85d3-02b08635bcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.504031 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39c380-580c-40fc-85d3-02b08635bcc5 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.504895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39c380-580c-40fc-85d3-02b08635bcc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:20.506487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39c380-580c-40fc-85d3-02b08635bcc5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:20.534848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 142/277] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:51:20 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:51:20.536818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f615-082c-42d1-9fcb-67dce4627479 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.537878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f615-082c-42d1-9fcb-67dce4627479 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.538457 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f615-082c-42d1-9fcb-67dce4627479 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.540817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 143/278] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:20.554018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e8379-157d-4ae3-8f09-e619dd90d80d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.554615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e8379-157d-4ae3-8f09-e619dd90d80d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.555092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e8379-157d-4ae3-8f09-e619dd90d80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.605722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-1b6e8379-157d-4ae3-8f09-e619dd90d80d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:20.607416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1b6e8379-157d-4ae3-8f09-e619dd90d80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:20.609067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2df4a-aea1-468a-be13-2ee03693e75f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:20.656605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67141191-8716-4876-9d24-058d85c7fb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:20.656605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67141191-8716-4876-9d24-058d85c7fb8b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.656935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67141191-8716-4876-9d24-058d85c7fb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.657206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67141191-8716-4876-9d24-058d85c7fb8b None admin] PATH_INFO: `/v3/users/ac901063af4943b581907a78731e1f65` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.675952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67141191-8716-4876-9d24-058d85c7fb8b None admin] RBAC: Authorizing `identity:get_user(user_id=ac901063af4943b581907a78731e1f65)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:20.677527 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67141191-8716-4876-9d24-058d85c7fb8b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:20.680936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 136/279] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:20 2022] GET /identity/v3/users/ac901063af4943b581907a78731e1f65 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:20.686834 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67141191-8716-4876-9d24-058d85c7fb8b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:20.758593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 144/280] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:20.772797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:20.786839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e8379-157d-4ae3-8f09-e619dd90d80d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:20.828744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67141191-8716-4876-9d24-058d85c7fb8b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:20.868227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb2fef-137c-40a2-a775-d0a78ef4af4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:20.868969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb2fef-137c-40a2-a775-d0a78ef4af4d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.869422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb2fef-137c-40a2-a775-d0a78ef4af4d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb2fef-137c-40a2-a775-d0a78ef4af4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.872725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb2fef-137c-40a2-a775-d0a78ef4af4d None admin] RBAC: Adding query filter params (role.id=61490cf1d7334421b84ff67552b79bfc, user.id=ac901063af4943b581907a78731e1f65, scope.system=all) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:20.872725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb2fef-137c-40a2-a775-d0a78ef4af4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:20.873114 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb2fef-137c-40a2-a775-d0a78ef4af4d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:20.881150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 137/281] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Jun 8 01:51:20 2022] GET /identity/v3/role_assignments?user.id=ac901063af4943b581907a78731e1f65&scope.system=all&role.id=61490cf1d7334421b84ff67552b79bfc => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:20.882467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afb881-183c-4ee1-ab76-2699522c091a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.882756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afb881-183c-4ee1-ab76-2699522c091a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.883096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afb881-183c-4ee1-ab76-2699522c091a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.885066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 138/282] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:20.891869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd6c5-1c23-45e5-92a5-d0f5acfce96f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.892165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd6c5-1c23-45e5-92a5-d0f5acfce96f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.893359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd6c5-1c23-45e5-92a5-d0f5acfce96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.914679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fd9fd6c5-1c23-45e5-92a5-d0f5acfce96f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:20.915305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:20.916314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fd9fd6c5-1c23-45e5-92a5-d0f5acfce96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:20.922434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e8379-157d-4ae3-8f09-e619dd90d80d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:20.979640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352db32-f343-4fc2-9bf4-bb27786db517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:20.980759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352db32-f343-4fc2-9bf4-bb27786db517 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:20.981309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352db32-f343-4fc2-9bf4-bb27786db517 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:20.981736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352db32-f343-4fc2-9bf4-bb27786db517 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:20.998390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352db32-f343-4fc2-9bf4-bb27786db517 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-1352db32-f343-4fc2-9bf4-bb27786db517 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:21.006591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:21.007746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:21.008652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 145/283] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:21.012265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 139/284] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:21.015645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352db32-f343-4fc2-9bf4-bb27786db517 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:21.037925 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fd6c5-1c23-45e5-92a5-d0f5acfce96f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:21.133889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:21.168337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352db32-f343-4fc2-9bf4-bb27786db517 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:21.190263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fd6c5-1c23-45e5-92a5-d0f5acfce96f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:21.217398 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1ff6f-6c94-4e80-9178-4e061f19cdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:21.218088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1ff6f-6c94-4e80-9178-4e061f19cdfd None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:21.218374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1ff6f-6c94-4e80-9178-4e061f19cdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:21.218650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1ff6f-6c94-4e80-9178-4e061f19cdfd None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:21.219419 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1ff6f-6c94-4e80-9178-4e061f19cdfd None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:21.219729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1ff6f-6c94-4e80-9178-4e061f19cdfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:21.221173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1ff6f-6c94-4e80-9178-4e061f19cdfd None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:21.242799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 146/285] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:21.249428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1ff6f-6c94-4e80-9178-4e061f19cdfd None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:21.262167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a543ae7-8fde-4095-adc2-3b26735f5b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:21.262910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543ae7-8fde-4095-adc2-3b26735f5b3c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:21.263195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543ae7-8fde-4095-adc2-3b26735f5b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:21.263470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543ae7-8fde-4095-adc2-3b26735f5b3c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:21.292821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:21.297059 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a543ae7-8fde-4095-adc2-3b26735f5b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-6a543ae7-8fde-4095-adc2-3b26735f5b3c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:21.305164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:21.306080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:21.307655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 140/286] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:21.317461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a543ae7-8fde-4095-adc2-3b26735f5b3c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:21.390806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1ff6f-6c94-4e80-9178-4e061f19cdfd None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:21.442265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440a67a-8464-462a-a4a1-15d95dc68d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:21.443158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440a67a-8464-462a-a4a1-15d95dc68d51 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:21.443448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440a67a-8464-462a-a4a1-15d95dc68d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:21.443729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440a67a-8464-462a-a4a1-15d95dc68d51 None admin] PATH_INFO: `/v3/users/4c1388585e3345cf993c6d5a3aa08b7f` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:21.448915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440a67a-8464-462a-a4a1-15d95dc68d51 None admin] RBAC: Authorizing `identity:get_user(user_id=4c1388585e3345cf993c6d5a3aa08b7f)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:21.450387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440a67a-8464-462a-a4a1-15d95dc68d51 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:21.452234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 147/287] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:21 2022] GET /identity/v3/users/4c1388585e3345cf993c6d5a3aa08b7f => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:21.457639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440a67a-8464-462a-a4a1-15d95dc68d51 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:21.561172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a543ae7-8fde-4095-adc2-3b26735f5b3c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:21.595958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:21.635666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:21.638777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4a6b3-0c1c-4bcb-894d-967ea2233063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:21.639523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4a6b3-0c1c-4bcb-894d-967ea2233063 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:21.639810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4a6b3-0c1c-4bcb-894d-967ea2233063 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:21.640239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4a6b3-0c1c-4bcb-894d-967ea2233063 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:21.641070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4a6b3-0c1c-4bcb-894d-967ea2233063 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:21.641391 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4a6b3-0c1c-4bcb-894d-967ea2233063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:21.646068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4a6b3-0c1c-4bcb-894d-967ea2233063 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:21.660440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 141/288] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:21.666974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4a6b3-0c1c-4bcb-894d-967ea2233063 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:21.669406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440a67a-8464-462a-a4a1-15d95dc68d51 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:21.742685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593655f-4d57-4d4b-b41b-3e6c4bb2daef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:21.743700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593655f-4d57-4d4b-b41b-3e6c4bb2daef None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:21.744360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593655f-4d57-4d4b-b41b-3e6c4bb2daef None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:21.745069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593655f-4d57-4d4b-b41b-3e6c4bb2daef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:21.822711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4a6b3-0c1c-4bcb-894d-967ea2233063 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:21.859429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1de3e0-31a8-4668-9801-f96e39790138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:21.860650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1de3e0-31a8-4668-9801-f96e39790138 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:21.861219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1de3e0-31a8-4668-9801-f96e39790138 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:21.861685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1de3e0-31a8-4668-9801-f96e39790138 None admin] PATH_INFO: `/v3/users/a75a6f58e11a4643a6539f43589c90a7` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:21.863082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1de3e0-31a8-4668-9801-f96e39790138 None admin] RBAC: Authorizing `identity:get_user(user_id=a75a6f58e11a4643a6539f43589c90a7)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:21.864758 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1de3e0-31a8-4668-9801-f96e39790138 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:21.866966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 142/289] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:21 2022] GET /identity/v3/users/a75a6f58e11a4643a6539f43589c90a7 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:21.873365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1de3e0-31a8-4668-9801-f96e39790138 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:21.902699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593655f-4d57-4d4b-b41b-3e6c4bb2daef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:21.904187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593655f-4d57-4d4b-b41b-3e6c4bb2daef None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:21.955453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:22.051925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:22.061535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1de3e0-31a8-4668-9801-f96e39790138 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:22.085887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 148/290] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:22.092165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593655f-4d57-4d4b-b41b-3e6c4bb2daef None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:22.133183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0585c-4d16-4bb3-b837-64edd333f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:22.134217 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0585c-4d16-4bb3-b837-64edd333f842 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:22.134546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0585c-4d16-4bb3-b837-64edd333f842 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:22.134887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0585c-4d16-4bb3-b837-64edd333f842 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:22.137108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0585c-4d16-4bb3-b837-64edd333f842 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:22.138717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0585c-4d16-4bb3-b837-64edd333f842 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:22.141005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 143/291] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:21 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:22.146441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0585c-4d16-4bb3-b837-64edd333f842 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:22.233664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593655f-4d57-4d4b-b41b-3e6c4bb2daef None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:22.273925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba02bf-3f0f-43a1-9f85-463e2601b250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:22.274827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba02bf-3f0f-43a1-9f85-463e2601b250 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:22.275601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba02bf-3f0f-43a1-9f85-463e2601b250 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:22.279151 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba02bf-3f0f-43a1-9f85-463e2601b250 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:22.280799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba02bf-3f0f-43a1-9f85-463e2601b250 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:22.282319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba02bf-3f0f-43a1-9f85-463e2601b250 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:22.284142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 149/292] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:22 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:22.290839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba02bf-3f0f-43a1-9f85-463e2601b250 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:22.346233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0585c-4d16-4bb3-b837-64edd333f842 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:22.379258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:22.390983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:22.421493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd0ad5-0691-4f89-92b1-fd981bf14ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:22.422401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd0ad5-0691-4f89-92b1-fd981bf14ff5 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:22.422683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd0ad5-0691-4f89-92b1-fd981bf14ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:22.422956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd0ad5-0691-4f89-92b1-fd981bf14ff5 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a/users/a75a6f58e11a4643a6539f43589c90a7/roles/e9245ca6f01f444dbbd7fe5e6121449c` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:22.425007 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd0ad5-0691-4f89-92b1-fd981bf14ff5 None admin] RBAC: Authorizing `identity:create_grant(project_id=defbc73a230d43b5afdb17c929c4e61a, user_id=a75a6f58e11a4643a6539f43589c90a7, role_id=e9245ca6f01f444dbbd7fe5e6121449c)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:22.426595 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd0ad5-0691-4f89-92b1-fd981bf14ff5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:22.453050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 144/293] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:51:22 2022] PUT /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a/users/a75a6f58e11a4643a6539f43589c90a7/roles/e9245ca6f01f444dbbd7fe5e6121449c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:22.465110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba02bf-3f0f-43a1-9f85-463e2601b250 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:22.523378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2d654-5169-4781-8016-682f63a4022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:22.524109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2d654-5169-4781-8016-682f63a4022e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:22.524418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2d654-5169-4781-8016-682f63a4022e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:22.524743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2d654-5169-4781-8016-682f63a4022e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:22.525935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2d654-5169-4781-8016-682f63a4022e None admin] RBAC: Adding query filter params (role.id=61490cf1d7334421b84ff67552b79bfc, scope.project.id=defbc73a230d43b5afdb17c929c4e61a, user.id=4c1388585e3345cf993c6d5a3aa08b7f) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:22.526252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2d654-5169-4781-8016-682f63a4022e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:22.527807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2d654-5169-4781-8016-682f63a4022e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:22.534587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 150/294] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:22 2022] GET /identity/v3/role_assignments?user.id=4c1388585e3345cf993c6d5a3aa08b7f&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=61490cf1d7334421b84ff67552b79bfc => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:23.287742 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894c7bb-ed04-4168-845b-92770a7a0b1a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:23.288858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894c7bb-ed04-4168-845b-92770a7a0b1a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:23.289350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894c7bb-ed04-4168-845b-92770a7a0b1a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:23.291751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 145/295] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:23.297518 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791551a8-665a-4aea-898d-4cf4d311231c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:23.298046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791551a8-665a-4aea-898d-4cf4d311231c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:23.298530 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791551a8-665a-4aea-898d-4cf4d311231c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:23.316028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-791551a8-665a-4aea-898d-4cf4d311231c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:23.317611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-791551a8-665a-4aea-898d-4cf4d311231c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:23.396115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 151/296] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:23.420326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894c7bb-ed04-4168-845b-92770a7a0b1a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:23.487039 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852aed0-02e3-4b7f-87cd-2e51c16e91fc None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:23.487660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852aed0-02e3-4b7f-87cd-2e51c16e91fc None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:23.488157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852aed0-02e3-4b7f-87cd-2e51c16e91fc None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:23.489759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 152/297] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:23.496538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a94db-0599-4119-920a-cdc7b4d29cbf None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:23.497134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a94db-0599-4119-920a-cdc7b4d29cbf None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:23.498951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a94db-0599-4119-920a-cdc7b4d29cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:23.518675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-077a94db-0599-4119-920a-cdc7b4d29cbf None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:23.519860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-077a94db-0599-4119-920a-cdc7b4d29cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:23.534483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894c7bb-ed04-4168-845b-92770a7a0b1a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:23.567902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:23.571214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4d5f7-2895-40c7-bf77-f54090854762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:23.572652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4d5f7-2895-40c7-bf77-f54090854762 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:23.572950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4d5f7-2895-40c7-bf77-f54090854762 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:23.573224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4d5f7-2895-40c7-bf77-f54090854762 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:23.574401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4d5f7-2895-40c7-bf77-f54090854762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:23.575866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4d5f7-2895-40c7-bf77-f54090854762 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:23.577705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 146/298] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:23.578724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315f2c2-038d-450c-b074-68cc228368f9 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:23.579011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315f2c2-038d-450c-b074-68cc228368f9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:23.579292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315f2c2-038d-450c-b074-68cc228368f9 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:23.580535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 147/299] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:23.585240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315f2c2-038d-450c-b074-68cc228368f9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:23.600437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 153/300] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:23.601725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a534bd4-2901-4cab-be04-c23b04302b1e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:23.602016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a534bd4-2901-4cab-be04-c23b04302b1e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:23.602300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a534bd4-2901-4cab-be04-c23b04302b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:23.628512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7a534bd4-2901-4cab-be04-c23b04302b1e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:23.630136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7a534bd4-2901-4cab-be04-c23b04302b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:23.636012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:23.704081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 154/301] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:23.705346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a534bd4-2901-4cab-be04-c23b04302b1e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:23.714095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315f2c2-038d-450c-b074-68cc228368f9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:23.752323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a04b28-3d6f-43f6-a8f5-d8d691d370b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:23.753328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a04b28-3d6f-43f6-a8f5-d8d691d370b7 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:23.753801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a04b28-3d6f-43f6-a8f5-d8d691d370b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:23.754248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a04b28-3d6f-43f6-a8f5-d8d691d370b7 None admin] PATH_INFO: `/v3/groups` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:23.755110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a04b28-3d6f-43f6-a8f5-d8d691d370b7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:23.756773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a04b28-3d6f-43f6-a8f5-d8d691d370b7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:23.776070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 148/302] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Jun 8 01:51:23 2022] POST /identity/v3/groups => generated 229 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:51:23.777124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a04b28-3d6f-43f6-a8f5-d8d691d370b7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:23.821014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a534bd4-2901-4cab-be04-c23b04302b1e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:23.831432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:23.865468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de213230-ce61-4aeb-a4c0-b5d2da59b3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:23.866635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de213230-ce61-4aeb-a4c0-b5d2da59b3ff None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:23.867103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de213230-ce61-4aeb-a4c0-b5d2da59b3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:23.867614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de213230-ce61-4aeb-a4c0-b5d2da59b3ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:23.881228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de213230-ce61-4aeb-a4c0-b5d2da59b3ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-de213230-ce61-4aeb-a4c0-b5d2da59b3ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:23.889809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:23.890793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:23.891601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:23.894176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 155/303] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:23.900639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de213230-ce61-4aeb-a4c0-b5d2da59b3ff None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:23.945899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a04b28-3d6f-43f6-a8f5-d8d691d370b7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:23.990242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:23.997838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a29ca-509a-4e6c-ad99-32b560affab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:23.999237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a29ca-509a-4e6c-ad99-32b560affab3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:23.999724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a29ca-509a-4e6c-ad99-32b560affab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:24.000202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a29ca-509a-4e6c-ad99-32b560affab3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:24.012911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a29ca-509a-4e6c-ad99-32b560affab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5a0a29ca-509a-4e6c-ad99-32b560affab3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:24.021017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:24.021891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:24.022678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:24.026384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 149/304] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:24.036527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de213230-ce61-4aeb-a4c0-b5d2da59b3ff None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:24.036703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a29ca-509a-4e6c-ad99-32b560affab3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:24.067642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969925a5-2f85-4b0c-a674-7b628a0495c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:24.068332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969925a5-2f85-4b0c-a674-7b628a0495c4 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:24.068628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969925a5-2f85-4b0c-a674-7b628a0495c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:24.068908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969925a5-2f85-4b0c-a674-7b628a0495c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:24.069774 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969925a5-2f85-4b0c-a674-7b628a0495c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:24.070083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969925a5-2f85-4b0c-a674-7b628a0495c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:24.071653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969925a5-2f85-4b0c-a674-7b628a0495c4 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:24.081392 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 156/305] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:24.086779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969925a5-2f85-4b0c-a674-7b628a0495c4 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:24.118997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:24.166868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a29ca-509a-4e6c-ad99-32b560affab3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:24.196023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969925a5-2f85-4b0c-a674-7b628a0495c4 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:24.209586 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02c312-4fad-4c46-84f4-1dab62c9d1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:24.210259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02c312-4fad-4c46-84f4-1dab62c9d1fc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:24.210543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02c312-4fad-4c46-84f4-1dab62c9d1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:24.210817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02c312-4fad-4c46-84f4-1dab62c9d1fc None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:24.211562 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02c312-4fad-4c46-84f4-1dab62c9d1fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:24.211870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02c312-4fad-4c46-84f4-1dab62c9d1fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:24.213536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02c312-4fad-4c46-84f4-1dab62c9d1fc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:24.222614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 150/306] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:24.227848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02c312-4fad-4c46-84f4-1dab62c9d1fc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:24.239570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682ca2c-9eb2-4121-b997-5565c918ebb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:24.241211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682ca2c-9eb2-4121-b997-5565c918ebb6 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:24.241778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682ca2c-9eb2-4121-b997-5565c918ebb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:24.242234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682ca2c-9eb2-4121-b997-5565c918ebb6 None admin] PATH_INFO: `/v3/users/823a1e0615c3432a951a617a56dfd103` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:24.258783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682ca2c-9eb2-4121-b997-5565c918ebb6 None admin] RBAC: Authorizing `identity:get_user(user_id=823a1e0615c3432a951a617a56dfd103)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:24.260473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682ca2c-9eb2-4121-b997-5565c918ebb6 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:24.262191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 157/307] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:24 2022] GET /identity/v3/users/823a1e0615c3432a951a617a56dfd103 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:24.267833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682ca2c-9eb2-4121-b997-5565c918ebb6 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:24.271290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:24.359985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02c312-4fad-4c46-84f4-1dab62c9d1fc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:24.378832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682ca2c-9eb2-4121-b997-5565c918ebb6 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:24.403728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141b7b5-e5bf-4496-9c31-a5efe9332c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:24.404826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:24.405274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141b7b5-e5bf-4496-9c31-a5efe9332c56 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:24.405274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141b7b5-e5bf-4496-9c31-a5efe9332c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:24.405539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141b7b5-e5bf-4496-9c31-a5efe9332c56 None admin] PATH_INFO: `/v3/users/ac901063af4943b581907a78731e1f65` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:24.407842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141b7b5-e5bf-4496-9c31-a5efe9332c56 None admin] RBAC: Authorizing `identity:get_user(user_id=ac901063af4943b581907a78731e1f65)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:24.409437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141b7b5-e5bf-4496-9c31-a5efe9332c56 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:24.416253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 151/308] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:24 2022] GET /identity/v3/users/ac901063af4943b581907a78731e1f65 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:24.417514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141b7b5-e5bf-4496-9c31-a5efe9332c56 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:24.423207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6397de-a8ea-45ea-ba65-bd8d8781c96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:24.423922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6397de-a8ea-45ea-ba65-bd8d8781c96a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:24.424205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6397de-a8ea-45ea-ba65-bd8d8781c96a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:24.424509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6397de-a8ea-45ea-ba65-bd8d8781c96a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:24.425342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6397de-a8ea-45ea-ba65-bd8d8781c96a None admin] RBAC: Adding query filter params (role.id=e9245ca6f01f444dbbd7fe5e6121449c, user.id=823a1e0615c3432a951a617a56dfd103, scope.system=all) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:24.425648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6397de-a8ea-45ea-ba65-bd8d8781c96a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:24.427088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6397de-a8ea-45ea-ba65-bd8d8781c96a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:24.439054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 158/309] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Jun 8 01:51:24 2022] GET /identity/v3/role_assignments?user.id=823a1e0615c3432a951a617a56dfd103&scope.system=all&role.id=e9245ca6f01f444dbbd7fe5e6121449c => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:24.538194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141b7b5-e5bf-4496-9c31-a5efe9332c56 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:24.546343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:24.576234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35a655-bc97-43b7-b0ed-dd9d090bab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:24.577152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a655-bc97-43b7-b0ed-dd9d090bab8e None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:24.577434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a655-bc97-43b7-b0ed-dd9d090bab8e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:24.577709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a655-bc97-43b7-b0ed-dd9d090bab8e None admin] PATH_INFO: `/v3/system/users/ac901063af4943b581907a78731e1f65/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:24.579277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35a655-bc97-43b7-b0ed-dd9d090bab8e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ac901063af4943b581907a78731e1f65, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:24.580716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35a655-bc97-43b7-b0ed-dd9d090bab8e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:24.594093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 152/310] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Jun 8 01:51:24 2022] PUT /identity/v3/system/users/ac901063af4943b581907a78731e1f65/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 01:51:25.034493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264f15d-648f-4d0e-a632-5be67172401b None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:25.034807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264f15d-648f-4d0e-a632-5be67172401b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:25.035058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264f15d-648f-4d0e-a632-5be67172401b None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:25.036876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 159/311] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:25.042228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba96c0-2635-403d-b9f8-d201078beb34 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:25.042521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba96c0-2635-403d-b9f8-d201078beb34 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:25.042805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba96c0-2635-403d-b9f8-d201078beb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:25.060211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-23ba96c0-2635-403d-b9f8-d201078beb34 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:25.061434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-23ba96c0-2635-403d-b9f8-d201078beb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:25.129599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 153/312] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:25.153532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264f15d-648f-4d0e-a632-5be67172401b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:25.267339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264f15d-648f-4d0e-a632-5be67172401b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:25.304472 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41e2bb-d4eb-4773-9933-60d59b0942bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:25.305614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41e2bb-d4eb-4773-9933-60d59b0942bf None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:25.306121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41e2bb-d4eb-4773-9933-60d59b0942bf None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:25.307118 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41e2bb-d4eb-4773-9933-60d59b0942bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:25.318265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41e2bb-d4eb-4773-9933-60d59b0942bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-0d41e2bb-d4eb-4773-9933-60d59b0942bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:25.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:25.327734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:25.328558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:25.330528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 160/313] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:25.336389 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba96c0-2635-403d-b9f8-d201078beb34 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:25.426701 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:25.439797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba96c0-2635-403d-b9f8-d201078beb34 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:25.476021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089d0ac-978c-434f-a1c1-bc4686523dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:25.476721 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089d0ac-978c-434f-a1c1-bc4686523dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:25.477003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089d0ac-978c-434f-a1c1-bc4686523dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:25.477277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089d0ac-978c-434f-a1c1-bc4686523dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:25.478015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089d0ac-978c-434f-a1c1-bc4686523dc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:25.478898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089d0ac-978c-434f-a1c1-bc4686523dc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:25.480275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089d0ac-978c-434f-a1c1-bc4686523dc5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:25.488644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 154/314] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:25.495093 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41e2bb-d4eb-4773-9933-60d59b0942bf None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:25.508845 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:25.610946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41e2bb-d4eb-4773-9933-60d59b0942bf None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:25.651051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eefa1a-31f5-4269-8e8e-fba2a009699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:25.652235 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eefa1a-31f5-4269-8e8e-fba2a009699b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:25.652721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eefa1a-31f5-4269-8e8e-fba2a009699b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:25.653175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eefa1a-31f5-4269-8e8e-fba2a009699b None admin] PATH_INFO: `/v3/users/a75a6f58e11a4643a6539f43589c90a7` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:25.654579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eefa1a-31f5-4269-8e8e-fba2a009699b None admin] RBAC: Authorizing `identity:get_user(user_id=a75a6f58e11a4643a6539f43589c90a7)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:25.656217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eefa1a-31f5-4269-8e8e-fba2a009699b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:25.658357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 161/315] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:25 2022] GET /identity/v3/users/a75a6f58e11a4643a6539f43589c90a7 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:25.663753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089d0ac-978c-434f-a1c1-bc4686523dc5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:25.767715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089d0ac-978c-434f-a1c1-bc4686523dc5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:25.806991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4ffc4-b285-43a2-9f44-adec581d0a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:25.807680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ffc4-b285-43a2-9f44-adec581d0a47 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:25.807963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ffc4-b285-43a2-9f44-adec581d0a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:25.808238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ffc4-b285-43a2-9f44-adec581d0a47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:25.829748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:25.881766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4ffc4-b285-43a2-9f44-adec581d0a47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:25.883167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4ffc4-b285-43a2-9f44-adec581d0a47 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:25.996296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 155/316] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:26.001501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eefa1a-31f5-4269-8e8e-fba2a009699b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:26.083139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:26.106336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eefa1a-31f5-4269-8e8e-fba2a009699b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:26.142899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db104908-8424-4136-a74d-4fbd6e8dae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:26.143691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db104908-8424-4136-a74d-4fbd6e8dae63 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:26.143979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db104908-8424-4136-a74d-4fbd6e8dae63 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:26.144260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db104908-8424-4136-a74d-4fbd6e8dae63 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:26.145490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db104908-8424-4136-a74d-4fbd6e8dae63 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:26.146957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db104908-8424-4136-a74d-4fbd6e8dae63 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:26.148056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2d4b2-0776-4ec4-8bb5-c721c6f654d6 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:26.148551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2d4b2-0776-4ec4-8bb5-c721c6f654d6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:26.148975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 162/317] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:26 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:26.149110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2d4b2-0776-4ec4-8bb5-c721c6f654d6 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:26.150730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 156/318] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:26.154625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db104908-8424-4136-a74d-4fbd6e8dae63 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:26.155743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de09b5-a9b3-4ac7-81f9-f19cb5038370 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:26.156188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de09b5-a9b3-4ac7-81f9-f19cb5038370 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:26.156661 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de09b5-a9b3-4ac7-81f9-f19cb5038370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:26.173963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-84de09b5-a9b3-4ac7-81f9-f19cb5038370 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:26.175373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-84de09b5-a9b3-4ac7-81f9-f19cb5038370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:26.235127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:26.261978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 157/319] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:26.270276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db104908-8424-4136-a74d-4fbd6e8dae63 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:26.285810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de09b5-a9b3-4ac7-81f9-f19cb5038370 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:26.315779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d736b6-99a4-4ff5-9851-5666e37e7284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:26.316908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d736b6-99a4-4ff5-9851-5666e37e7284 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:26.317196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d736b6-99a4-4ff5-9851-5666e37e7284 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:26.317484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d736b6-99a4-4ff5-9851-5666e37e7284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:26.318619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d736b6-99a4-4ff5-9851-5666e37e7284 None admin] RBAC: Adding query filter params (role.id=e9245ca6f01f444dbbd7fe5e6121449c, scope.project.id=defbc73a230d43b5afdb17c929c4e61a, user.id=a75a6f58e11a4643a6539f43589c90a7) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:26.318939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d736b6-99a4-4ff5-9851-5666e37e7284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:26.320851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d736b6-99a4-4ff5-9851-5666e37e7284 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:26.332959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 163/320] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:26 2022] GET /identity/v3/role_assignments?user.id=a75a6f58e11a4643a6539f43589c90a7&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=e9245ca6f01f444dbbd7fe5e6121449c => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:26.407793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de09b5-a9b3-4ac7-81f9-f19cb5038370 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:26.449466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695c0d6-f52c-4e7d-ab92-3b7c49f44dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:26.451143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695c0d6-f52c-4e7d-ab92-3b7c49f44dbe None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:26.451664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695c0d6-f52c-4e7d-ab92-3b7c49f44dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:26.452159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695c0d6-f52c-4e7d-ab92-3b7c49f44dbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:26.453714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695c0d6-f52c-4e7d-ab92-3b7c49f44dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:26.455430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695c0d6-f52c-4e7d-ab92-3b7c49f44dbe None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:26.457751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 158/321] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:26.463741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d736b6-99a4-4ff5-9851-5666e37e7284 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:26.478388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:26.585037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d736b6-99a4-4ff5-9851-5666e37e7284 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:26.628301 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96f29c-f8c3-4c24-972b-26265ae1586a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:26.629057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96f29c-f8c3-4c24-972b-26265ae1586a None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:26.629384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96f29c-f8c3-4c24-972b-26265ae1586a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:26.629668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96f29c-f8c3-4c24-972b-26265ae1586a None admin] PATH_INFO: `/v3/groups` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:26.630378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96f29c-f8c3-4c24-972b-26265ae1586a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:26.631913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96f29c-f8c3-4c24-972b-26265ae1586a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:26.651256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 164/322] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Jun 8 01:51:26 2022] POST /identity/v3/groups => generated 226 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:51:26.875854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2087c-16f9-4c0e-9ded-e3f2c6761fe7 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:26.876140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2087c-16f9-4c0e-9ded-e3f2c6761fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:26.876484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2087c-16f9-4c0e-9ded-e3f2c6761fe7 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:26.877880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 159/323] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:26.883239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c1feb-0156-4a37-8e6d-c622b4b653f2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:26.883542 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c1feb-0156-4a37-8e6d-c622b4b653f2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:26.883830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c1feb-0156-4a37-8e6d-c622b4b653f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:26.905191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-117c1feb-0156-4a37-8e6d-c622b4b653f2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:26.906387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-117c1feb-0156-4a37-8e6d-c622b4b653f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:26.981808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:26.987761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 165/324] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:27.011452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2087c-16f9-4c0e-9ded-e3f2c6761fe7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:27.031519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c008-3bd9-4675-9d75-51452c692783 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:27.031805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c008-3bd9-4675-9d75-51452c692783 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:27.032082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c008-3bd9-4675-9d75-51452c692783 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:27.033767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 166/325] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:51:27.039147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846eb1e-57c5-45c9-b8fc-9dfb681e8a32 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:27.039433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846eb1e-57c5-45c9-b8fc-9dfb681e8a32 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:27.039715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846eb1e-57c5-45c9-b8fc-9dfb681e8a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:27.057870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-c846eb1e-57c5-45c9-b8fc-9dfb681e8a32 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:27.059069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c846eb1e-57c5-45c9-b8fc-9dfb681e8a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:27.074683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:27.138078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2087c-16f9-4c0e-9ded-e3f2c6761fe7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:27.142303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 167/326] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:27.167686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846eb1e-57c5-45c9-b8fc-9dfb681e8a32 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:27.180819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623fd3d-99a1-4053-9834-035cc204cc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:27.182513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623fd3d-99a1-4053-9834-035cc204cc0e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:27.183332 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623fd3d-99a1-4053-9834-035cc204cc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:27.183991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623fd3d-99a1-4053-9834-035cc204cc0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:27.199919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623fd3d-99a1-4053-9834-035cc204cc0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0623fd3d-99a1-4053-9834-035cc204cc0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:27.214504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:27.215686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:27.216766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:27.219198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 160/327] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:27.228417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623fd3d-99a1-4053-9834-035cc204cc0e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:27.288438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846eb1e-57c5-45c9-b8fc-9dfb681e8a32 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:27.320505 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:27.330577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880b0bb-4b75-47b1-8af5-a1f0fbae385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:27.331402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880b0bb-4b75-47b1-8af5-a1f0fbae385c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:27.331867 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880b0bb-4b75-47b1-8af5-a1f0fbae385c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:27.332139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880b0bb-4b75-47b1-8af5-a1f0fbae385c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:27.343886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880b0bb-4b75-47b1-8af5-a1f0fbae385c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:27.345894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623fd3d-99a1-4053-9834-035cc204cc0e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-4880b0bb-4b75-47b1-8af5-a1f0fbae385c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:27.351673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:27.352593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:27.353737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 168/328] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:27.361576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880b0bb-4b75-47b1-8af5-a1f0fbae385c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:27.363890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:27.412751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3de4a-a0e8-4480-b522-7ca8fcfb0f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:27.413350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3de4a-a0e8-4480-b522-7ca8fcfb0f41 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:27.413630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3de4a-a0e8-4480-b522-7ca8fcfb0f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:27.413904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3de4a-a0e8-4480-b522-7ca8fcfb0f41 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:27.414649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3de4a-a0e8-4480-b522-7ca8fcfb0f41 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:27.414953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3de4a-a0e8-4480-b522-7ca8fcfb0f41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:27.417869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3de4a-a0e8-4480-b522-7ca8fcfb0f41 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:27.428346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 161/329] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:27.435071 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3de4a-a0e8-4480-b522-7ca8fcfb0f41 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:27.484506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880b0bb-4b75-47b1-8af5-a1f0fbae385c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:27.533111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ed570-6a5f-4a8a-b629-778a3ff29e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:27.533805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ed570-6a5f-4a8a-b629-778a3ff29e12 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:27.534411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ed570-6a5f-4a8a-b629-778a3ff29e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:27.534696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ed570-6a5f-4a8a-b629-778a3ff29e12 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:27.535762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ed570-6a5f-4a8a-b629-778a3ff29e12 None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:27.536079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ed570-6a5f-4a8a-b629-778a3ff29e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:27.538691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ed570-6a5f-4a8a-b629-778a3ff29e12 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:27.551243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 169/330] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:27.556843 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ed570-6a5f-4a8a-b629-778a3ff29e12 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:27.574754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3de4a-a0e8-4480-b522-7ca8fcfb0f41 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:27.640268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96e5d0-91af-471a-b4da-fc2e7ff20c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:27.641256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96e5d0-91af-471a-b4da-fc2e7ff20c72 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:27.641562 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96e5d0-91af-471a-b4da-fc2e7ff20c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:27.641934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96e5d0-91af-471a-b4da-fc2e7ff20c72 None admin] PATH_INFO: `/v3/users/823a1e0615c3432a951a617a56dfd103` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:27.644079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96e5d0-91af-471a-b4da-fc2e7ff20c72 None admin] RBAC: Authorizing `identity:get_user(user_id=823a1e0615c3432a951a617a56dfd103)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:27.645585 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96e5d0-91af-471a-b4da-fc2e7ff20c72 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:27.647237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 162/331] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:27 2022] GET /identity/v3/users/823a1e0615c3432a951a617a56dfd103 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:27.650970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:27.656208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96e5d0-91af-471a-b4da-fc2e7ff20c72 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:27.738359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:27.750600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ed570-6a5f-4a8a-b629-778a3ff29e12 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:27.805910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96e5d0-91af-471a-b4da-fc2e7ff20c72 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:27.845266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f671ce-8195-4ebd-abab-dc46afb3e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:27.846660 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f671ce-8195-4ebd-abab-dc46afb3e387 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:27.847035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f671ce-8195-4ebd-abab-dc46afb3e387 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:27.847456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f671ce-8195-4ebd-abab-dc46afb3e387 None admin] PATH_INFO: `/v3/system/users/823a1e0615c3432a951a617a56dfd103/roles/e9245ca6f01f444dbbd7fe5e6121449c` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:27.849030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f671ce-8195-4ebd-abab-dc46afb3e387 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=823a1e0615c3432a951a617a56dfd103, role_id=e9245ca6f01f444dbbd7fe5e6121449c)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:27.850675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f671ce-8195-4ebd-abab-dc46afb3e387 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:27.856497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce4acc-4842-417e-a043-d608b92011c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:27.857406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce4acc-4842-417e-a043-d608b92011c3 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:27.857737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce4acc-4842-417e-a043-d608b92011c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:27.857962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce4acc-4842-417e-a043-d608b92011c3 None admin] PATH_INFO: `/v3/users/ac901063af4943b581907a78731e1f65` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:27.859365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce4acc-4842-417e-a043-d608b92011c3 None admin] RBAC: Authorizing `identity:get_user(user_id=ac901063af4943b581907a78731e1f65)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:27.862669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce4acc-4842-417e-a043-d608b92011c3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:27.862669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 170/332] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:27 2022] GET /identity/v3/users/ac901063af4943b581907a78731e1f65 => generated 293 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:27.864954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 163/333] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Jun 8 01:51:27 2022] PUT /identity/v3/system/users/823a1e0615c3432a951a617a56dfd103/roles/e9245ca6f01f444dbbd7fe5e6121449c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:27.869408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce4acc-4842-417e-a043-d608b92011c3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:28.006396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce4acc-4842-417e-a043-d608b92011c3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:28.030082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:28.047452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d05a4-a44e-4f11-8747-6a59b8be65b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:28.048184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d05a4-a44e-4f11-8747-6a59b8be65b0 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:28.048492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d05a4-a44e-4f11-8747-6a59b8be65b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:28.048826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d05a4-a44e-4f11-8747-6a59b8be65b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:28.049675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d05a4-a44e-4f11-8747-6a59b8be65b0 None admin] RBAC: Adding query filter params (role.id=61490cf1d7334421b84ff67552b79bfc, user.id=ac901063af4943b581907a78731e1f65, scope.system=all) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:28.050037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d05a4-a44e-4f11-8747-6a59b8be65b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:28.051516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d05a4-a44e-4f11-8747-6a59b8be65b0 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:28.068291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 171/334] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Jun 8 01:51:27 2022] GET /identity/v3/role_assignments?user.id=ac901063af4943b581907a78731e1f65&scope.system=all&role.id=61490cf1d7334421b84ff67552b79bfc => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:29.860249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59100440-10ed-4326-ae8c-1a157318badb None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:29.861301 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59100440-10ed-4326-ae8c-1a157318badb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:29.861797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59100440-10ed-4326-ae8c-1a157318badb None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:29.863486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 164/335] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:29.882384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4c9c-af30-4b64-bf96-83f2587fa2cc None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:29.883359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4c9c-af30-4b64-bf96-83f2587fa2cc None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:29.883843 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4c9c-af30-4b64-bf96-83f2587fa2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:29.905928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-578f4c9c-af30-4b64-bf96-83f2587fa2cc None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:29.907499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-578f4c9c-af30-4b64-bf96-83f2587fa2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:30.008304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 172/336] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:30.040159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59100440-10ed-4326-ae8c-1a157318badb None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:30.183778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59100440-10ed-4326-ae8c-1a157318badb None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:30.186043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:30.233521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60412921-9ad2-4556-9579-56d6cc824849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:30.234730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60412921-9ad2-4556-9579-56d6cc824849 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:30.235222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60412921-9ad2-4556-9579-56d6cc824849 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:30.235686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60412921-9ad2-4556-9579-56d6cc824849 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:30.252176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60412921-9ad2-4556-9579-56d6cc824849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-60412921-9ad2-4556-9579-56d6cc824849 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:30.260535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:30.261550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:30.262333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:30.267490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 165/337] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:30.270705 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4c9c-af30-4b64-bf96-83f2587fa2cc None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:30.331832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba902e-6d8e-41f9-b7a1-dd4d05958579 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:30.332119 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba902e-6d8e-41f9-b7a1-dd4d05958579 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:30.335605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba902e-6d8e-41f9-b7a1-dd4d05958579 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:30.335605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 166/338] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:30.341757 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd69172-45cc-4abe-b7de-0ef6737b3603 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:30.342139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd69172-45cc-4abe-b7de-0ef6737b3603 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:30.342486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd69172-45cc-4abe-b7de-0ef6737b3603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:30.364522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9cd69172-45cc-4abe-b7de-0ef6737b3603 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:30.366156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9cd69172-45cc-4abe-b7de-0ef6737b3603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:30.448671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4c9c-af30-4b64-bf96-83f2587fa2cc None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:30.463459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:30.496062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 167/339] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:30.521105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc4a18-d1d5-4925-bdbe-cdce3f397ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:30.521793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc4a18-d1d5-4925-bdbe-cdce3f397ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:30.522083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc4a18-d1d5-4925-bdbe-cdce3f397ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:30.522365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc4a18-d1d5-4925-bdbe-cdce3f397ce0 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:30.523135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc4a18-d1d5-4925-bdbe-cdce3f397ce0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:30.523443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc4a18-d1d5-4925-bdbe-cdce3f397ce0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:30.525421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc4a18-d1d5-4925-bdbe-cdce3f397ce0 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:30.535985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 173/340] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:30.542315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd69172-45cc-4abe-b7de-0ef6737b3603 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:30.547686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc4a18-d1d5-4925-bdbe-cdce3f397ce0 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:30.730663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:30.747878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc4a18-d1d5-4925-bdbe-cdce3f397ce0 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:30.750828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd69172-45cc-4abe-b7de-0ef6737b3603 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:30.804934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f29e1f-f04e-44d9-b0f2-8ca62cbc67fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:30.805743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f29e1f-f04e-44d9-b0f2-8ca62cbc67fc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:30.806119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f29e1f-f04e-44d9-b0f2-8ca62cbc67fc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:30.806399 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f29e1f-f04e-44d9-b0f2-8ca62cbc67fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:30.807614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2eaa5-4073-4990-9421-c053b8d4d572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:30.808697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2eaa5-4073-4990-9421-c053b8d4d572 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:30.809463 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2eaa5-4073-4990-9421-c053b8d4d572 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:30.809943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2eaa5-4073-4990-9421-c053b8d4d572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:30.819769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f29e1f-f04e-44d9-b0f2-8ca62cbc67fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-71f29e1f-f04e-44d9-b0f2-8ca62cbc67fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:30.830984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:30.831929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:30.832767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 174/341] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:30.833422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ea347-419f-4364-9893-fdccde420c91 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:30.834335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ea347-419f-4364-9893-fdccde420c91 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:30.834335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ea347-419f-4364-9893-fdccde420c91 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:30.839815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 175/342] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:30.840952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ea347-419f-4364-9893-fdccde420c91 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:30.877255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:30.929423 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2eaa5-4073-4990-9421-c053b8d4d572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:30.933072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2eaa5-4073-4990-9421-c053b8d4d572 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:30.983213 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ea347-419f-4364-9893-fdccde420c91 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:31.023437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776eb3f-22ce-4632-b189-bfe2f9efd32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:31.024158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776eb3f-22ce-4632-b189-bfe2f9efd32a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:31.024546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776eb3f-22ce-4632-b189-bfe2f9efd32a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:31.025352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776eb3f-22ce-4632-b189-bfe2f9efd32a None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:31.026262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776eb3f-22ce-4632-b189-bfe2f9efd32a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:31.026569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776eb3f-22ce-4632-b189-bfe2f9efd32a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:31.027963 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776eb3f-22ce-4632-b189-bfe2f9efd32a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:31.037576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 176/343] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:31.042569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf58aa-0c34-4ea2-8bac-61f36c3b58a2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:31.042569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf58aa-0c34-4ea2-8bac-61f36c3b58a2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:31.042569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf58aa-0c34-4ea2-8bac-61f36c3b58a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:31.059545 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:31.059854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3abf58aa-0c34-4ea2-8bac-61f36c3b58a2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:31.061644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3abf58aa-0c34-4ea2-8bac-61f36c3b58a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:31.076871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 168/344] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:31.080887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f859e-4ec0-4de0-bbee-de01bb533bbf None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:31.081311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f859e-4ec0-4de0-bbee-de01bb533bbf None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:31.081777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f859e-4ec0-4de0-bbee-de01bb533bbf None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:31.083354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 169/345] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:31.085392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f859e-4ec0-4de0-bbee-de01bb533bbf None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:31.155371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 177/346] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:31.156232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf58aa-0c34-4ea2-8bac-61f36c3b58a2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:31.184467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:31.239952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f859e-4ec0-4de0-bbee-de01bb533bbf None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:31.277787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a23897-aada-4abc-ab44-1b836918ecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:31.278452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a23897-aada-4abc-ab44-1b836918ecc2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:31.278732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a23897-aada-4abc-ab44-1b836918ecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:31.279008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a23897-aada-4abc-ab44-1b836918ecc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:31.283508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf58aa-0c34-4ea2-8bac-61f36c3b58a2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:31.322112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863302ab-a956-4cac-bb17-54bb6c103900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:31.323408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863302ab-a956-4cac-bb17-54bb6c103900 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:31.323890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863302ab-a956-4cac-bb17-54bb6c103900 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:31.324409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863302ab-a956-4cac-bb17-54bb6c103900 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:31.325927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863302ab-a956-4cac-bb17-54bb6c103900 None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:31.327639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863302ab-a956-4cac-bb17-54bb6c103900 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:31.330030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 178/347] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:31 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:31.331509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89282e-300f-4675-a7da-19b59f25efaf None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:31.331993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89282e-300f-4675-a7da-19b59f25efaf None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:31.336824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89282e-300f-4675-a7da-19b59f25efaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:31.343801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:31.354509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a23897-aada-4abc-ab44-1b836918ecc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:31.356100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a23897-aada-4abc-ab44-1b836918ecc2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:31.357206 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-0c89282e-300f-4675-a7da-19b59f25efaf None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:31.358608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0c89282e-300f-4675-a7da-19b59f25efaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:31.436232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 179/348] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:31.437645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89282e-300f-4675-a7da-19b59f25efaf None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:31.461352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:31.477489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 170/349] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:31.478467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a23897-aada-4abc-ab44-1b836918ecc2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:31.557967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89282e-300f-4675-a7da-19b59f25efaf None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:31.598045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a23897-aada-4abc-ab44-1b836918ecc2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:31.598749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe37656-1f53-4ffd-b4a1-0e76cdc5ad1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:31.601167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe37656-1f53-4ffd-b4a1-0e76cdc5ad1d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:31.601294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:31.601796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe37656-1f53-4ffd-b4a1-0e76cdc5ad1d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:31.602298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe37656-1f53-4ffd-b4a1-0e76cdc5ad1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:31.615161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe37656-1f53-4ffd-b4a1-0e76cdc5ad1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-bbe37656-1f53-4ffd-b4a1-0e76cdc5ad1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:31.625247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:31.626202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:31.627092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:31.629706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 180/350] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:31.630699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe37656-1f53-4ffd-b4a1-0e76cdc5ad1d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:31.644454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e7c0a-c797-4a31-8bda-2ae88813658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:31.645374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e7c0a-c797-4a31-8bda-2ae88813658c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:31.645660 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e7c0a-c797-4a31-8bda-2ae88813658c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:31.645930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e7c0a-c797-4a31-8bda-2ae88813658c None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:31.653823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e7c0a-c797-4a31-8bda-2ae88813658c None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:31.655521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e7c0a-c797-4a31-8bda-2ae88813658c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:31.658535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 171/351] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:31 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:31.663811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e7c0a-c797-4a31-8bda-2ae88813658c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:31.724795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:31.746982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe37656-1f53-4ffd-b4a1-0e76cdc5ad1d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:31.778129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e7c0a-c797-4a31-8bda-2ae88813658c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:31.785022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86b24f-6b03-4889-80a8-83db126a2739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:31.786394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86b24f-6b03-4889-80a8-83db126a2739 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:31.786899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86b24f-6b03-4889-80a8-83db126a2739 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:31.787376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86b24f-6b03-4889-80a8-83db126a2739 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:31.800521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86b24f-6b03-4889-80a8-83db126a2739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-4a86b24f-6b03-4889-80a8-83db126a2739 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:31.813139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:31.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:31.814866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:31.816949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09360a0a-abac-4dd6-8bda-980c274b188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:31.817563 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 181/352] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:31.818197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09360a0a-abac-4dd6-8bda-980c274b188d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:31.818487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09360a0a-abac-4dd6-8bda-980c274b188d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:31.818890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09360a0a-abac-4dd6-8bda-980c274b188d None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:31.819188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86b24f-6b03-4889-80a8-83db126a2739 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:31.820204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09360a0a-abac-4dd6-8bda-980c274b188d None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:31.822222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09360a0a-abac-4dd6-8bda-980c274b188d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:31.824187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 172/353] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:31 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:31.828542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09360a0a-abac-4dd6-8bda-980c274b188d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:31.860753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:31.955197 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86b24f-6b03-4889-80a8-83db126a2739 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:31.980341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09360a0a-abac-4dd6-8bda-980c274b188d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:32.000837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cb1b5-a051-426d-a453-35c77cc42c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:32.002050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cb1b5-a051-426d-a453-35c77cc42c33 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.002593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cb1b5-a051-426d-a453-35c77cc42c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.003018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cb1b5-a051-426d-a453-35c77cc42c33 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.003922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cb1b5-a051-426d-a453-35c77cc42c33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:32.004426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cb1b5-a051-426d-a453-35c77cc42c33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:32.006097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cb1b5-a051-426d-a453-35c77cc42c33 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:32.015026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 182/354] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:32.015835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cb1b5-a051-426d-a453-35c77cc42c33 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:32.019424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:32.022437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98577688-bee5-4895-ba67-fa52c5344ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:32.023137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98577688-bee5-4895-ba67-fa52c5344ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.023398 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98577688-bee5-4895-ba67-fa52c5344ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.023727 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98577688-bee5-4895-ba67-fa52c5344ed8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.025313 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98577688-bee5-4895-ba67-fa52c5344ed8 None admin] RBAC: Adding query filter params (group.id=d4941b980ca24cca8e0c12ef50bce322, role.id=61490cf1d7334421b84ff67552b79bfc, scope.project.id=faa8ebf890584746af20166b89a7e162) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:32.025496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98577688-bee5-4895-ba67-fa52c5344ed8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:32.026853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98577688-bee5-4895-ba67-fa52c5344ed8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:32.033863 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 173/355] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jun 8 01:51:31 2022] GET /identity/v3/role_assignments?group.id=d4941b980ca24cca8e0c12ef50bce322&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=61490cf1d7334421b84ff67552b79bfc => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:32.036430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6840bba-a7cf-44b0-af73-cc5da0bb1bc9 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.036430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6840bba-a7cf-44b0-af73-cc5da0bb1bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.036430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6840bba-a7cf-44b0-af73-cc5da0bb1bc9 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.037169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 174/356] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:32.039099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6840bba-a7cf-44b0-af73-cc5da0bb1bc9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:32.161604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cb1b5-a051-426d-a453-35c77cc42c33 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:32.163850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:32.193112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6840bba-a7cf-44b0-af73-cc5da0bb1bc9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:32.214079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8269a52-9265-4637-a109-fe903238773e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:32.214781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8269a52-9265-4637-a109-fe903238773e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.215066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8269a52-9265-4637-a109-fe903238773e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.215345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8269a52-9265-4637-a109-fe903238773e None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.216127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8269a52-9265-4637-a109-fe903238773e None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:32.216740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8269a52-9265-4637-a109-fe903238773e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:32.218685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8269a52-9265-4637-a109-fe903238773e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:32.232782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 183/357] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:32.234396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b55e99-cb48-4bc5-9a4e-cc1685beb4b2 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.234687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b55e99-cb48-4bc5-9a4e-cc1685beb4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.234966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b55e99-cb48-4bc5-9a4e-cc1685beb4b2 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.236309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 184/358] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:32.239080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b55e99-cb48-4bc5-9a4e-cc1685beb4b2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:32.244319 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff826e-1fb7-4451-bcbe-25e75ee8a39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:32.245343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff826e-1fb7-4451-bcbe-25e75ee8a39a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.245724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff826e-1fb7-4451-bcbe-25e75ee8a39a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.246112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff826e-1fb7-4451-bcbe-25e75ee8a39a None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.247427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff826e-1fb7-4451-bcbe-25e75ee8a39a None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:32.248935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff826e-1fb7-4451-bcbe-25e75ee8a39a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:32.250619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 175/359] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:32 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:32.252778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5742a-8f7e-4618-87f4-f821d4de5c92 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.253062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5742a-8f7e-4618-87f4-f821d4de5c92 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.253334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5742a-8f7e-4618-87f4-f821d4de5c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.272181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-77f5742a-8f7e-4618-87f4-f821d4de5c92 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:32.274021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-77f5742a-8f7e-4618-87f4-f821d4de5c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:32.332745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:32.366058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 176/360] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:32.367379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5742a-8f7e-4618-87f4-f821d4de5c92 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:32.389179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b55e99-cb48-4bc5-9a4e-cc1685beb4b2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:32.433136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6138163-077e-4c3a-a989-029b4de504ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:32.433834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6138163-077e-4c3a-a989-029b4de504ca None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.434127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6138163-077e-4c3a-a989-029b4de504ca None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.434408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6138163-077e-4c3a-a989-029b4de504ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.473113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:32.496443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5742a-8f7e-4618-87f4-f821d4de5c92 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:32.535876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6138163-077e-4c3a-a989-029b4de504ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:32.537604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6138163-077e-4c3a-a989-029b4de504ca None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:32.544598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b36eee-91a1-4de9-b8b8-21c39f915be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:32.545558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b36eee-91a1-4de9-b8b8-21c39f915be4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.546033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b36eee-91a1-4de9-b8b8-21c39f915be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.546984 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b36eee-91a1-4de9-b8b8-21c39f915be4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.614110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:32.636211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b36eee-91a1-4de9-b8b8-21c39f915be4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:32.638113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b36eee-91a1-4de9-b8b8-21c39f915be4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:32.665955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 185/361] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:32.667364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5eb39-5949-47a7-a84e-233a9f673285 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.667688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5eb39-5949-47a7-a84e-233a9f673285 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.667943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5eb39-5949-47a7-a84e-233a9f673285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.685640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-c3b5eb39-5949-47a7-a84e-233a9f673285 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:32.686771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c3b5eb39-5949-47a7-a84e-233a9f673285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:32.743958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:32.770450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 177/362] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:32.771030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b36eee-91a1-4de9-b8b8-21c39f915be4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:32.777237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 186/363] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:32.778099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5eb39-5949-47a7-a84e-233a9f673285 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:32.898767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:32.902372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b36eee-91a1-4de9-b8b8-21c39f915be4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:32.908609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5eb39-5949-47a7-a84e-233a9f673285 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:32.943011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78ab97-29e3-4ba5-95ce-0272c504832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:32.943416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78ab97-29e3-4ba5-95ce-0272c504832d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.943706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78ab97-29e3-4ba5-95ce-0272c504832d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.943989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78ab97-29e3-4ba5-95ce-0272c504832d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.944863 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78ab97-29e3-4ba5-95ce-0272c504832d None admin] RBAC: Adding query filter params (group.id=d4941b980ca24cca8e0c12ef50bce322, role.id=c45c9440d2b8447ab50b814c6be35bf1, scope.project.id=faa8ebf890584746af20166b89a7e162) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:32.945183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78ab97-29e3-4ba5-95ce-0272c504832d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:32.947155 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78ab97-29e3-4ba5-95ce-0272c504832d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:32.960912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 178/364] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jun 8 01:51:32 2022] GET /identity/v3/role_assignments?group.id=d4941b980ca24cca8e0c12ef50bce322&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=c45c9440d2b8447ab50b814c6be35bf1 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:32.961599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78ab97-29e3-4ba5-95ce-0272c504832d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:32.970295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd4015-1598-4f65-92ed-86e8c3a6b63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:32.972067 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd4015-1598-4f65-92ed-86e8c3a6b63f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:32.972815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd4015-1598-4f65-92ed-86e8c3a6b63f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:32.973520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd4015-1598-4f65-92ed-86e8c3a6b63f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:32.991094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd4015-1598-4f65-92ed-86e8c3a6b63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-dcbd4015-1598-4f65-92ed-86e8c3a6b63f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:33.002115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:33.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 08 01:51:33.004336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:33.007731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 187/365] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:33.007888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd4015-1598-4f65-92ed-86e8c3a6b63f None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:33.037787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:33.122395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78ab97-29e3-4ba5-95ce-0272c504832d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:33.165961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd4015-1598-4f65-92ed-86e8c3a6b63f None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:33.173184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e98fb5-6d67-4d5e-82c6-4810e8742522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:33.174071 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e98fb5-6d67-4d5e-82c6-4810e8742522 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:33.174355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e98fb5-6d67-4d5e-82c6-4810e8742522 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:33.174626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e98fb5-6d67-4d5e-82c6-4810e8742522 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:33.177187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e98fb5-6d67-4d5e-82c6-4810e8742522 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:33.178758 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e98fb5-6d67-4d5e-82c6-4810e8742522 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:33.183177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 179/366] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:32 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:33.183177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e98fb5-6d67-4d5e-82c6-4810e8742522 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:33.217125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba191fe-0078-45d7-af67-5c4fc01c1c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:33.219127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba191fe-0078-45d7-af67-5c4fc01c1c04 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:33.220148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba191fe-0078-45d7-af67-5c4fc01c1c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:33.220464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba191fe-0078-45d7-af67-5c4fc01c1c04 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:33.221911 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba191fe-0078-45d7-af67-5c4fc01c1c04 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:33.224547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba191fe-0078-45d7-af67-5c4fc01c1c04 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:33.227412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 188/367] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:33 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:33.229452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba191fe-0078-45d7-af67-5c4fc01c1c04 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:33.264787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:33.326903 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e98fb5-6d67-4d5e-82c6-4810e8742522 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:33.349349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:33.366379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba191fe-0078-45d7-af67-5c4fc01c1c04 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:33.375234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebfe1-5db2-4d87-90ef-cb4386795c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:33.376042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebfe1-5db2-4d87-90ef-cb4386795c59 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:33.376328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebfe1-5db2-4d87-90ef-cb4386795c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:33.376971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebfe1-5db2-4d87-90ef-cb4386795c59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:33.390703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ebfe1-5db2-4d87-90ef-cb4386795c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a38ebfe1-5db2-4d87-90ef-cb4386795c59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:33.398897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:33.400355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:33.401177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:33.401775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 180/368] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:33.403146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebfe1-5db2-4d87-90ef-cb4386795c59 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:33.417711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b0a6d-e2b3-4a3e-a09c-eebbaa0e4322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:33.419062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b0a6d-e2b3-4a3e-a09c-eebbaa0e4322 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:33.419648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b0a6d-e2b3-4a3e-a09c-eebbaa0e4322 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:33.420192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b0a6d-e2b3-4a3e-a09c-eebbaa0e4322 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:33.421117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b0a6d-e2b3-4a3e-a09c-eebbaa0e4322 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:33.421821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b0a6d-e2b3-4a3e-a09c-eebbaa0e4322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:33.424102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b0a6d-e2b3-4a3e-a09c-eebbaa0e4322 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:33.440575 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 189/369] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:33.441444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b0a6d-e2b3-4a3e-a09c-eebbaa0e4322 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:33.527205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebfe1-5db2-4d87-90ef-cb4386795c59 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:33.560669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:33.568823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9571e-7814-4d06-9fcc-8ad37988bfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:33.569777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9571e-7814-4d06-9fcc-8ad37988bfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:33.570136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9571e-7814-4d06-9fcc-8ad37988bfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:33.570488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9571e-7814-4d06-9fcc-8ad37988bfd4 None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:33.572631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9571e-7814-4d06-9fcc-8ad37988bfd4 None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:33.574219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9571e-7814-4d06-9fcc-8ad37988bfd4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:33.580422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 181/370] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:33 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:33.580422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9571e-7814-4d06-9fcc-8ad37988bfd4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:33.587853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b0a6d-e2b3-4a3e-a09c-eebbaa0e4322 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:33.633419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:33.635099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff30c39-5beb-42a9-94ff-42896a4e9bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:33.636866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff30c39-5beb-42a9-94ff-42896a4e9bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:33.638103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff30c39-5beb-42a9-94ff-42896a4e9bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:33.638585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff30c39-5beb-42a9-94ff-42896a4e9bc9 None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:33.640373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff30c39-5beb-42a9-94ff-42896a4e9bc9 None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:33.642578 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff30c39-5beb-42a9-94ff-42896a4e9bc9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:33.644568 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 190/371] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:33 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:33.645381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff30c39-5beb-42a9-94ff-42896a4e9bc9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:33.705157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9571e-7814-4d06-9fcc-8ad37988bfd4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:33.745083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077197b5-f8c0-4403-b519-69842294f1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:33.745798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077197b5-f8c0-4403-b519-69842294f1bd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:33.746292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077197b5-f8c0-4403-b519-69842294f1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:33.746396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077197b5-f8c0-4403-b519-69842294f1bd None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:33.747096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077197b5-f8c0-4403-b519-69842294f1bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:33.747395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077197b5-f8c0-4403-b519-69842294f1bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:33.749866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077197b5-f8c0-4403-b519-69842294f1bd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:33.761192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 182/372] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:33.761627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077197b5-f8c0-4403-b519-69842294f1bd None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:33.775543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff30c39-5beb-42a9-94ff-42896a4e9bc9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:33.830102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97569e66-3c44-4c85-986b-980fb546631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:33.831962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569e66-3c44-4c85-986b-980fb546631e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:33.831962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569e66-3c44-4c85-986b-980fb546631e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:33.831962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569e66-3c44-4c85-986b-980fb546631e None admin] PATH_INFO: `/v3/users/823a1e0615c3432a951a617a56dfd103` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:33.833092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97569e66-3c44-4c85-986b-980fb546631e None admin] RBAC: Authorizing `identity:get_user(user_id=823a1e0615c3432a951a617a56dfd103)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:33.834665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97569e66-3c44-4c85-986b-980fb546631e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:33.840418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 191/373] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:51:33 2022] GET /identity/v3/users/823a1e0615c3432a951a617a56dfd103 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:33.840418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97569e66-3c44-4c85-986b-980fb546631e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:33.872272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:33.901635 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077197b5-f8c0-4403-b519-69842294f1bd None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:33.914518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:33.946194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db27a7-1866-48df-81ce-45d5e5008563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:33.947572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db27a7-1866-48df-81ce-45d5e5008563 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:33.947857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db27a7-1866-48df-81ce-45d5e5008563 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:33.948183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db27a7-1866-48df-81ce-45d5e5008563 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:33.949000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db27a7-1866-48df-81ce-45d5e5008563 None admin] RBAC: Adding query filter params (group.id=d4941b980ca24cca8e0c12ef50bce322, role.id=c45c9440d2b8447ab50b814c6be35bf1, scope.project.id=defbc73a230d43b5afdb17c929c4e61a) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:33.949311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db27a7-1866-48df-81ce-45d5e5008563 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:33.950839 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db27a7-1866-48df-81ce-45d5e5008563 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:33.961002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 183/374] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jun 8 01:51:33 2022] GET /identity/v3/role_assignments?group.id=d4941b980ca24cca8e0c12ef50bce322&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=c45c9440d2b8447ab50b814c6be35bf1 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:33.961002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db27a7-1866-48df-81ce-45d5e5008563 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:34.001517 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97569e66-3c44-4c85-986b-980fb546631e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:34.078423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df048da-8838-408b-8e1f-3cf4af0f26ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:34.079191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df048da-8838-408b-8e1f-3cf4af0f26ac None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:34.079379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df048da-8838-408b-8e1f-3cf4af0f26ac None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:34.079659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df048da-8838-408b-8e1f-3cf4af0f26ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:34.080587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df048da-8838-408b-8e1f-3cf4af0f26ac None admin] RBAC: Adding query filter params (group.id=d4941b980ca24cca8e0c12ef50bce322, role.id=61490cf1d7334421b84ff67552b79bfc, scope.project.id=defbc73a230d43b5afdb17c929c4e61a) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:34.080855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df048da-8838-408b-8e1f-3cf4af0f26ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:34.082519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df048da-8838-408b-8e1f-3cf4af0f26ac None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:34.101261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 192/375] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jun 8 01:51:33 2022] GET /identity/v3/role_assignments?group.id=d4941b980ca24cca8e0c12ef50bce322&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=61490cf1d7334421b84ff67552b79bfc => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:34.102193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df048da-8838-408b-8e1f-3cf4af0f26ac None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:34.176023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db27a7-1866-48df-81ce-45d5e5008563 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:34.224880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1009b-bd97-4a40-af03-ca43c3ec7ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:34.225575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1009b-bd97-4a40-af03-ca43c3ec7ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:34.225862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1009b-bd97-4a40-af03-ca43c3ec7ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:34.226143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1009b-bd97-4a40-af03-ca43c3ec7ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:34.299146 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:34.324347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df048da-8838-408b-8e1f-3cf4af0f26ac None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:34.328963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:34.390383 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180dd53-d258-4468-a4dc-f50d45c7124a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:34.390963 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180dd53-d258-4468-a4dc-f50d45c7124a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:34.391247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180dd53-d258-4468-a4dc-f50d45c7124a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:34.391522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180dd53-d258-4468-a4dc-f50d45c7124a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:34.392371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180dd53-d258-4468-a4dc-f50d45c7124a None admin] RBAC: Adding query filter params (role.id=e9245ca6f01f444dbbd7fe5e6121449c, user.id=823a1e0615c3432a951a617a56dfd103, scope.system=all) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:34.392711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180dd53-d258-4468-a4dc-f50d45c7124a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:34.394188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180dd53-d258-4468-a4dc-f50d45c7124a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:34.413584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1009b-bd97-4a40-af03-ca43c3ec7ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:34.415079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1009b-bd97-4a40-af03-ca43c3ec7ad3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:34.417371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 193/376] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Jun 8 01:51:34 2022] GET /identity/v3/role_assignments?user.id=823a1e0615c3432a951a617a56dfd103&scope.system=all&role.id=e9245ca6f01f444dbbd7fe5e6121449c => generated 510 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:34.619638 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 184/377] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:34.626711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180dd53-d258-4468-a4dc-f50d45c7124a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:34.711176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65761740-4f5f-4592-b456-23612fda3543 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:34.711456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65761740-4f5f-4592-b456-23612fda3543 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:34.711632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65761740-4f5f-4592-b456-23612fda3543 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:34.714286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 185/378] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:34.718639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66238d4-09a8-46b4-988f-2c7070e2e23e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:34.718927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66238d4-09a8-46b4-988f-2c7070e2e23e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:34.719208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66238d4-09a8-46b4-988f-2c7070e2e23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:34.750480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c66238d4-09a8-46b4-988f-2c7070e2e23e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:34.751674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c66238d4-09a8-46b4-988f-2c7070e2e23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:34.789449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180dd53-d258-4468-a4dc-f50d45c7124a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:34.852302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818a2fd-dad9-41cb-946c-2009064b0148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:34.853466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818a2fd-dad9-41cb-946c-2009064b0148 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:34.853749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818a2fd-dad9-41cb-946c-2009064b0148 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:34.854019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818a2fd-dad9-41cb-946c-2009064b0148 None admin] PATH_INFO: `/v3/projects/d34d18aa450b41d2ac1001da2d9a5476` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:34.855421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818a2fd-dad9-41cb-946c-2009064b0148 None admin] RBAC: Authorizing `identity:get_project(project_id=d34d18aa450b41d2ac1001da2d9a5476)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:34.856909 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818a2fd-dad9-41cb-946c-2009064b0148 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:34.858599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 194/379] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:34 2022] GET /identity/v3/projects/d34d18aa450b41d2ac1001da2d9a5476 => generated 344 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:34.864459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818a2fd-dad9-41cb-946c-2009064b0148 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:34.873378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 186/380] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:34.914081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:34.919068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66238d4-09a8-46b4-988f-2c7070e2e23e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:34.922493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:35.001993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818a2fd-dad9-41cb-946c-2009064b0148 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:35.042676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dba89-9938-47da-9503-5148ec44057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:35.043711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dba89-9938-47da-9503-5148ec44057b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:35.044166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dba89-9938-47da-9503-5148ec44057b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:35.044621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dba89-9938-47da-9503-5148ec44057b None admin] PATH_INFO: `/v3/groups/ff0446e46bde4cb892e4a5b48bebe14b` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:35.053006 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537dba89-9938-47da-9503-5148ec44057b None admin] RBAC: Authorizing `identity:get_group(group_id=ff0446e46bde4cb892e4a5b48bebe14b)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:35.054681 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537dba89-9938-47da-9503-5148ec44057b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:35.057215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 195/381] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:34 2022] GET /identity/v3/groups/ff0446e46bde4cb892e4a5b48bebe14b => generated 229 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:35.066299 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dba89-9938-47da-9503-5148ec44057b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:35.111615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66238d4-09a8-46b4-988f-2c7070e2e23e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:35.193179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490cd61-1cfc-4db0-a205-cfa5c3a79d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:35.193934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490cd61-1cfc-4db0-a205-cfa5c3a79d5e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:35.194218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490cd61-1cfc-4db0-a205-cfa5c3a79d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:35.194488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490cd61-1cfc-4db0-a205-cfa5c3a79d5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:35.206773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:35.221615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490cd61-1cfc-4db0-a205-cfa5c3a79d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-9490cd61-1cfc-4db0-a205-cfa5c3a79d5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:35.230295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:35.231424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:35.233182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:35.237218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 187/382] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:35.245208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dba89-9938-47da-9503-5148ec44057b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:35.245989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490cd61-1cfc-4db0-a205-cfa5c3a79d5e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:35.286294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485699a1-88a6-41c9-9c97-d4f2f49acdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:35.287288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485699a1-88a6-41c9-9c97-d4f2f49acdeb None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:35.287804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485699a1-88a6-41c9-9c97-d4f2f49acdeb None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:35.288295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485699a1-88a6-41c9-9c97-d4f2f49acdeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:35.289434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485699a1-88a6-41c9-9c97-d4f2f49acdeb None admin] RBAC: Adding query filter params (group.id=ff0446e46bde4cb892e4a5b48bebe14b, role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=d34d18aa450b41d2ac1001da2d9a5476) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:35.289955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485699a1-88a6-41c9-9c97-d4f2f49acdeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:35.292452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485699a1-88a6-41c9-9c97-d4f2f49acdeb None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:35.299490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 196/383] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jun 8 01:51:35 2022] GET /identity/v3/role_assignments?group.id=ff0446e46bde4cb892e4a5b48bebe14b&scope.project.id=d34d18aa450b41d2ac1001da2d9a5476&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 260 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:35.385962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:35.459003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490cd61-1cfc-4db0-a205-cfa5c3a79d5e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:35.517799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d136d9-ae64-4bb8-84f1-10f4e630d43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:35.518542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d136d9-ae64-4bb8-84f1-10f4e630d43d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:35.519028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d136d9-ae64-4bb8-84f1-10f4e630d43d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:35.519524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d136d9-ae64-4bb8-84f1-10f4e630d43d None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:35.520387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d136d9-ae64-4bb8-84f1-10f4e630d43d None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:35.520733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d136d9-ae64-4bb8-84f1-10f4e630d43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:35.522277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d136d9-ae64-4bb8-84f1-10f4e630d43d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:35.538109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 188/384] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:35.546803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485699a1-88a6-41c9-9c97-d4f2f49acdeb None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:35.725952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485699a1-88a6-41c9-9c97-d4f2f49acdeb None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:35.769361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fc1f9-2523-46ee-8ff3-9d40093ad9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:35.770286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fc1f9-2523-46ee-8ff3-9d40093ad9ea None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:35.770605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fc1f9-2523-46ee-8ff3-9d40093ad9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:35.770884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fc1f9-2523-46ee-8ff3-9d40093ad9ea None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:35.772091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fc1f9-2523-46ee-8ff3-9d40093ad9ea None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:35.773567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fc1f9-2523-46ee-8ff3-9d40093ad9ea None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:35.775830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 197/385] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:35 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:35.782450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d136d9-ae64-4bb8-84f1-10f4e630d43d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:35.852784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2331367-52ba-433a-9887-efc740a4dd13 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:35.852958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2331367-52ba-433a-9887-efc740a4dd13 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:35.853234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2331367-52ba-433a-9887-efc740a4dd13 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:35.856937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 198/386] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:35.862250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b51bd5-8b4d-4cee-97c9-796843360a4f None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:35.862643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b51bd5-8b4d-4cee-97c9-796843360a4f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:35.862916 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b51bd5-8b4d-4cee-97c9-796843360a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:35.876110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:35.885788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e5b51bd5-8b4d-4cee-97c9-796843360a4f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:35.887005 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e5b51bd5-8b4d-4cee-97c9-796843360a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:35.984517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d136d9-ae64-4bb8-84f1-10f4e630d43d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:35.992470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 199/387] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:36.021617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:36.022723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b51bd5-8b4d-4cee-97c9-796843360a4f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:36.047270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab4349-7030-463d-b51b-64873d96ff3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:36.048159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab4349-7030-463d-b51b-64873d96ff3e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:36.048452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab4349-7030-463d-b51b-64873d96ff3e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:36.048731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab4349-7030-463d-b51b-64873d96ff3e None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:36.050050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab4349-7030-463d-b51b-64873d96ff3e None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:36.051495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab4349-7030-463d-b51b-64873d96ff3e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:36.053227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 189/388] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:35 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:36.063640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab4349-7030-463d-b51b-64873d96ff3e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:36.252650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab4349-7030-463d-b51b-64873d96ff3e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:36.262878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b51bd5-8b4d-4cee-97c9-796843360a4f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:36.309604 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762b531-020e-4bd4-a024-4bbf9ea22de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:36.310505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762b531-020e-4bd4-a024-4bbf9ea22de1 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:36.310784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762b531-020e-4bd4-a024-4bbf9ea22de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:36.311060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762b531-020e-4bd4-a024-4bbf9ea22de1 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162/groups/d4941b980ca24cca8e0c12ef50bce322/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:36.314018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762b531-020e-4bd4-a024-4bbf9ea22de1 None admin] RBAC: Authorizing `identity:create_grant(project_id=faa8ebf890584746af20166b89a7e162, group_id=d4941b980ca24cca8e0c12ef50bce322, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:36.315501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762b531-020e-4bd4-a024-4bbf9ea22de1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:36.330452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba65c3-8d1d-44a3-89db-1e0fb216eb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:36.331682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba65c3-8d1d-44a3-89db-1e0fb216eb0b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:36.332565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba65c3-8d1d-44a3-89db-1e0fb216eb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:36.332880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba65c3-8d1d-44a3-89db-1e0fb216eb0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:36.341018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:36.344349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 190/389] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Jun 8 01:51:36 2022] PUT /identity/v3/projects/faa8ebf890584746af20166b89a7e162/groups/d4941b980ca24cca8e0c12ef50bce322/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:36.370050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba65c3-8d1d-44a3-89db-1e0fb216eb0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-a7ba65c3-8d1d-44a3-89db-1e0fb216eb0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:36.386938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:36.388256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:36.389506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:36.390895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 200/390] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:36.395678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762b531-020e-4bd4-a024-4bbf9ea22de1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:36.478572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:36.549659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762b531-020e-4bd4-a024-4bbf9ea22de1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:36.598526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050a98f-4bb0-4b15-b493-df13c9a5f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:36.599219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050a98f-4bb0-4b15-b493-df13c9a5f6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:36.599502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050a98f-4bb0-4b15-b493-df13c9a5f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:36.599772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050a98f-4bb0-4b15-b493-df13c9a5f6f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:36.600563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050a98f-4bb0-4b15-b493-df13c9a5f6f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:36.600871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050a98f-4bb0-4b15-b493-df13c9a5f6f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:36.602278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050a98f-4bb0-4b15-b493-df13c9a5f6f6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:36.616509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 191/391] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:36.621504 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba65c3-8d1d-44a3-89db-1e0fb216eb0b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:36.762889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba65c3-8d1d-44a3-89db-1e0fb216eb0b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:36.808602 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8572ce-4445-475c-8fff-06e6da785d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:36.809418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8572ce-4445-475c-8fff-06e6da785d1f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:36.809705 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8572ce-4445-475c-8fff-06e6da785d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:36.809980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8572ce-4445-475c-8fff-06e6da785d1f None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:36.811208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8572ce-4445-475c-8fff-06e6da785d1f None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:36.812724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8572ce-4445-475c-8fff-06e6da785d1f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:36.814855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 201/392] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:36 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:36.820142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050a98f-4bb0-4b15-b493-df13c9a5f6f6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:36.983705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050a98f-4bb0-4b15-b493-df13c9a5f6f6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:37.003359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:37.031674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2e319-85c4-45cb-b32a-ecb99fdc7555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:37.033160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2e319-85c4-45cb-b32a-ecb99fdc7555 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:37.033634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2e319-85c4-45cb-b32a-ecb99fdc7555 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:37.034128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2e319-85c4-45cb-b32a-ecb99fdc7555 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:37.039449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2e319-85c4-45cb-b32a-ecb99fdc7555 None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:37.041310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2e319-85c4-45cb-b32a-ecb99fdc7555 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:37.043435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 192/393] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:36 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:37.049122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8572ce-4445-475c-8fff-06e6da785d1f None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:37.135829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:37.168992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180eb45-ba86-42a7-90b5-93daca31b9c0 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:37.169441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180eb45-ba86-42a7-90b5-93daca31b9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:37.169598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180eb45-ba86-42a7-90b5-93daca31b9c0 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:37.171269 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 193/394] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:37.177750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227437d7-6a20-46bc-b1a7-0a84c1cf0a09 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:37.178039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227437d7-6a20-46bc-b1a7-0a84c1cf0a09 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:37.178316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227437d7-6a20-46bc-b1a7-0a84c1cf0a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:37.205376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-227437d7-6a20-46bc-b1a7-0a84c1cf0a09 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:37.206597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-227437d7-6a20-46bc-b1a7-0a84c1cf0a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:37.211175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8572ce-4445-475c-8fff-06e6da785d1f None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:37.264953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a8a9-572a-4b84-9274-d4e3a9199fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:37.265898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a8a9-572a-4b84-9274-d4e3a9199fe3 None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:37.266183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a8a9-572a-4b84-9274-d4e3a9199fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:37.266459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033a8a9-572a-4b84-9274-d4e3a9199fe3 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162/groups/d4941b980ca24cca8e0c12ef50bce322/roles/c45c9440d2b8447ab50b814c6be35bf1` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:37.270889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033a8a9-572a-4b84-9274-d4e3a9199fe3 None admin] RBAC: Authorizing `identity:create_grant(project_id=faa8ebf890584746af20166b89a7e162, group_id=d4941b980ca24cca8e0c12ef50bce322, role_id=c45c9440d2b8447ab50b814c6be35bf1)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:37.272332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033a8a9-572a-4b84-9274-d4e3a9199fe3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:37.304920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 202/395] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Jun 8 01:51:37 2022] PUT /identity/v3/projects/faa8ebf890584746af20166b89a7e162/groups/d4941b980ca24cca8e0c12ef50bce322/roles/c45c9440d2b8447ab50b814c6be35bf1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:37.322974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 194/396] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:37.359205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a8a9-572a-4b84-9274-d4e3a9199fe3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:37.497792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033a8a9-572a-4b84-9274-d4e3a9199fe3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:37.526614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:37.541759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea569dd7-f809-4beb-93ec-3eba8b9724e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:37.542666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea569dd7-f809-4beb-93ec-3eba8b9724e1 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:37.543016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea569dd7-f809-4beb-93ec-3eba8b9724e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:37.543477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea569dd7-f809-4beb-93ec-3eba8b9724e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:37.559727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea569dd7-f809-4beb-93ec-3eba8b9724e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-ea569dd7-f809-4beb-93ec-3eba8b9724e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:37.568254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:37.569242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:37.570018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:37.573102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 203/397] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:37.578141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227437d7-6a20-46bc-b1a7-0a84c1cf0a09 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:37.589917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a9b4-00fd-4421-b363-342016c99f36 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:37.590228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a9b4-00fd-4421-b363-342016c99f36 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:37.590463 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67a9b4-00fd-4421-b363-342016c99f36 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:37.591840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 204/398] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:37.598338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb988e-8fa5-4a8d-a05e-f2031b588562 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:37.598636 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb988e-8fa5-4a8d-a05e-f2031b588562 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:37.598892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb988e-8fa5-4a8d-a05e-f2031b588562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:37.633421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-57eb988e-8fa5-4a8d-a05e-f2031b588562 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:37.635553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-57eb988e-8fa5-4a8d-a05e-f2031b588562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:37.718431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227437d7-6a20-46bc-b1a7-0a84c1cf0a09 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:37.721751 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 205/399] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:37.736890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:37.748734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb988e-8fa5-4a8d-a05e-f2031b588562 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:37.781049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7f5f7-fd14-45ac-a7bb-682d02aa3d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:37.782079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f5f7-fd14-45ac-a7bb-682d02aa3d94 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:37.782367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f5f7-fd14-45ac-a7bb-682d02aa3d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:37.782647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7f5f7-fd14-45ac-a7bb-682d02aa3d94 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:37.783702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7f5f7-fd14-45ac-a7bb-682d02aa3d94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:37.784013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7f5f7-fd14-45ac-a7bb-682d02aa3d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:37.785909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7f5f7-fd14-45ac-a7bb-682d02aa3d94 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:37.802955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 195/400] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:37.813941 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7f5f7-fd14-45ac-a7bb-682d02aa3d94 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:37.935869 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb988e-8fa5-4a8d-a05e-f2031b588562 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:37.967862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:38.005170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c28f51-f8ee-4d77-b981-6e6b954c2e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:38.006390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c28f51-f8ee-4d77-b981-6e6b954c2e82 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:38.006895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c28f51-f8ee-4d77-b981-6e6b954c2e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:38.007546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c28f51-f8ee-4d77-b981-6e6b954c2e82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:38.010230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7f5f7-fd14-45ac-a7bb-682d02aa3d94 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:38.031970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c28f51-f8ee-4d77-b981-6e6b954c2e82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-39c28f51-f8ee-4d77-b981-6e6b954c2e82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:38.042486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:38.043401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:38.044191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:38.049167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 206/401] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:38.059613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c28f51-f8ee-4d77-b981-6e6b954c2e82 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:38.097503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaacbfa-0583-4f22-b3ea-195c6e55ca08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:38.098405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaacbfa-0583-4f22-b3ea-195c6e55ca08 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:38.098686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaacbfa-0583-4f22-b3ea-195c6e55ca08 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:38.098963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaacbfa-0583-4f22-b3ea-195c6e55ca08 None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:38.100596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaacbfa-0583-4f22-b3ea-195c6e55ca08 None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:38.102097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaacbfa-0583-4f22-b3ea-195c6e55ca08 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:38.111430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 196/402] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:37 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:38.112431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaacbfa-0583-4f22-b3ea-195c6e55ca08 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:38.202490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:38.277492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaacbfa-0583-4f22-b3ea-195c6e55ca08 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:38.278994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c28f51-f8ee-4d77-b981-6e6b954c2e82 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:38.338968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae4385-097d-4322-9914-dca6e573747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:38.339849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae4385-097d-4322-9914-dca6e573747d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:38.340134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae4385-097d-4322-9914-dca6e573747d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:38.340443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae4385-097d-4322-9914-dca6e573747d None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:38.341900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae4385-097d-4322-9914-dca6e573747d None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:38.343828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae4385-097d-4322-9914-dca6e573747d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:38.346246 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 197/403] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:38 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:38.354172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae4385-097d-4322-9914-dca6e573747d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:38.356406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2d4f0-0358-4f89-9c96-d5d164d7224b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:38.358003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2d4f0-0358-4f89-9c96-d5d164d7224b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:38.360314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2d4f0-0358-4f89-9c96-d5d164d7224b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:38.360772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2d4f0-0358-4f89-9c96-d5d164d7224b None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:38.361671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2d4f0-0358-4f89-9c96-d5d164d7224b None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:38.362119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2d4f0-0358-4f89-9c96-d5d164d7224b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:38.364482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2d4f0-0358-4f89-9c96-d5d164d7224b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:38.381604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 207/404] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:38.388883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2d4f0-0358-4f89-9c96-d5d164d7224b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:38.463469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:38.523440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae4385-097d-4322-9914-dca6e573747d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:38.537365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:38.547190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2d4f0-0358-4f89-9c96-d5d164d7224b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:38.568148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02daab2-70aa-49a6-bad9-09b03bd80673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:38.569198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02daab2-70aa-49a6-bad9-09b03bd80673 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:38.569484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02daab2-70aa-49a6-bad9-09b03bd80673 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:38.569761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02daab2-70aa-49a6-bad9-09b03bd80673 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a/groups/d4941b980ca24cca8e0c12ef50bce322/roles/c45c9440d2b8447ab50b814c6be35bf1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:38.572917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02daab2-70aa-49a6-bad9-09b03bd80673 None admin] RBAC: Authorizing `identity:create_grant(project_id=defbc73a230d43b5afdb17c929c4e61a, group_id=d4941b980ca24cca8e0c12ef50bce322, role_id=c45c9440d2b8447ab50b814c6be35bf1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:38.574542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02daab2-70aa-49a6-bad9-09b03bd80673 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:38.588831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f95ce-1250-461e-a559-3c9c37797e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:38.590178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f95ce-1250-461e-a559-3c9c37797e6a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:38.596428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 198/405] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Jun 8 01:51:38 2022] PUT /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a/groups/d4941b980ca24cca8e0c12ef50bce322/roles/c45c9440d2b8447ab50b814c6be35bf1 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:38.596428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288c13b-1782-483f-9097-88ad34b6eae0 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:38.596428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288c13b-1782-483f-9097-88ad34b6eae0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:38.596428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288c13b-1782-483f-9097-88ad34b6eae0 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:38.597289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f95ce-1250-461e-a559-3c9c37797e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:38.597926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f95ce-1250-461e-a559-3c9c37797e6a None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:38.598028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 199/406] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:38.599396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43137905-e44c-4f5b-920d-d6220981fd0a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:38.599685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43137905-e44c-4f5b-920d-d6220981fd0a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:38.599959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43137905-e44c-4f5b-920d-d6220981fd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:38.600234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f95ce-1250-461e-a559-3c9c37797e6a None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:38.602157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f95ce-1250-461e-a559-3c9c37797e6a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:38.610028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 208/407] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:38 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:38.611141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f95ce-1250-461e-a559-3c9c37797e6a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:38.621090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-43137905-e44c-4f5b-920d-d6220981fd0a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:38.622679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-43137905-e44c-4f5b-920d-d6220981fd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:38.707129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 200/408] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:38.734414 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43137905-e44c-4f5b-920d-d6220981fd0a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:38.829779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f95ce-1250-461e-a559-3c9c37797e6a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:38.838685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:38.873668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:38.877339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43137905-e44c-4f5b-920d-d6220981fd0a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:38.916366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790d2ba-25fc-4dc2-8c75-71e10579330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:38.916922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790d2ba-25fc-4dc2-8c75-71e10579330d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:38.917217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790d2ba-25fc-4dc2-8c75-71e10579330d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:38.917493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790d2ba-25fc-4dc2-8c75-71e10579330d None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:38.918734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790d2ba-25fc-4dc2-8c75-71e10579330d None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:38.926156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790d2ba-25fc-4dc2-8c75-71e10579330d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:38.931230 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 209/409] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:38 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:38.937638 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fcaa3-03db-4e02-abc3-e1930031c853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:38.937638 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fcaa3-03db-4e02-abc3-e1930031c853 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:38.937638 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fcaa3-03db-4e02-abc3-e1930031c853 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:38.937638 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fcaa3-03db-4e02-abc3-e1930031c853 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:38.938189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790d2ba-25fc-4dc2-8c75-71e10579330d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:38.955005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84fcaa3-03db-4e02-abc3-e1930031c853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-c84fcaa3-03db-4e02-abc3-e1930031c853 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:38.966469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:38.967363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:38.968550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 201/410] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:51:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:38.973680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fcaa3-03db-4e02-abc3-e1930031c853 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:39.124217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790d2ba-25fc-4dc2-8c75-71e10579330d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:39.168939 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fcaa3-03db-4e02-abc3-e1930031c853 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:39.184521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09693d-a02b-4a77-85ca-b2ba80390604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:39.185578 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09693d-a02b-4a77-85ca-b2ba80390604 None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.185864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09693d-a02b-4a77-85ca-b2ba80390604 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.186137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09693d-a02b-4a77-85ca-b2ba80390604 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a/groups/d4941b980ca24cca8e0c12ef50bce322/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:39.188720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09693d-a02b-4a77-85ca-b2ba80390604 None admin] RBAC: Authorizing `identity:create_grant(project_id=defbc73a230d43b5afdb17c929c4e61a, group_id=d4941b980ca24cca8e0c12ef50bce322, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:39.190394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09693d-a02b-4a77-85ca-b2ba80390604 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:39.212263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58ad60-47e4-44de-8b19-ec05799b5977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:39.213416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58ad60-47e4-44de-8b19-ec05799b5977 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.213933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58ad60-47e4-44de-8b19-ec05799b5977 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.214490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58ad60-47e4-44de-8b19-ec05799b5977 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:39.215593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58ad60-47e4-44de-8b19-ec05799b5977 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:39.216131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58ad60-47e4-44de-8b19-ec05799b5977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:39.216532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 210/411] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Jun 8 01:51:38 2022] PUT /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a/groups/d4941b980ca24cca8e0c12ef50bce322/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:39.218103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4a123-3b36-406e-aac1-0d73daad5f72 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.218322 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58ad60-47e4-44de-8b19-ec05799b5977 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:39.218421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4a123-3b36-406e-aac1-0d73daad5f72 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.218715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4a123-3b36-406e-aac1-0d73daad5f72 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:39.220622 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 211/412] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:39.229613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 202/413] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:39.232091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0987e-5044-4ab3-b23e-78d25beae4d0 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.232386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0987e-5044-4ab3-b23e-78d25beae4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.232682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0987e-5044-4ab3-b23e-78d25beae4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:39.234940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58ad60-47e4-44de-8b19-ec05799b5977 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:39.265289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:39.269660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-dfb0987e-5044-4ab3-b23e-78d25beae4d0 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:39.271231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-dfb0987e-5044-4ab3-b23e-78d25beae4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:39.310446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:39.356232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 212/414] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:39.361720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58ad60-47e4-44de-8b19-ec05799b5977 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:39.380460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0987e-5044-4ab3-b23e-78d25beae4d0 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:39.402377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d08e3b-7fed-44f1-ae9c-a894a667b785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:39.403781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d08e3b-7fed-44f1-ae9c-a894a667b785 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.404318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d08e3b-7fed-44f1-ae9c-a894a667b785 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.404808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d08e3b-7fed-44f1-ae9c-a894a667b785 None admin] PATH_INFO: `/v3/groups/ff0446e46bde4cb892e4a5b48bebe14b` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:39.406261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d08e3b-7fed-44f1-ae9c-a894a667b785 None admin] RBAC: Authorizing `identity:get_group(group_id=ff0446e46bde4cb892e4a5b48bebe14b)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:39.408127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d08e3b-7fed-44f1-ae9c-a894a667b785 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:39.410676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 203/415] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:39 2022] GET /identity/v3/groups/ff0446e46bde4cb892e4a5b48bebe14b => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:39.416497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d08e3b-7fed-44f1-ae9c-a894a667b785 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:39.562810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:39.565027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0987e-5044-4ab3-b23e-78d25beae4d0 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:39.590783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d08e3b-7fed-44f1-ae9c-a894a667b785 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:39.638426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69215bad-3ee0-4c72-a34b-33d5f925dce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:39.639165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69215bad-3ee0-4c72-a34b-33d5f925dce5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.639443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69215bad-3ee0-4c72-a34b-33d5f925dce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.639715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69215bad-3ee0-4c72-a34b-33d5f925dce5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:39.653893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1051d-6397-466c-beec-fae2687227e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:39.654857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1051d-6397-466c-beec-fae2687227e8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.655141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1051d-6397-466c-beec-fae2687227e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.655413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1051d-6397-466c-beec-fae2687227e8 None admin] PATH_INFO: `/v3/projects/d34d18aa450b41d2ac1001da2d9a5476` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:39.657148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1051d-6397-466c-beec-fae2687227e8 None admin] RBAC: Authorizing `identity:get_project(project_id=d34d18aa450b41d2ac1001da2d9a5476)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:39.658637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1051d-6397-466c-beec-fae2687227e8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:39.663677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 204/416] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:39 2022] GET /identity/v3/projects/d34d18aa450b41d2ac1001da2d9a5476 => generated 344 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:39.664262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69215bad-3ee0-4c72-a34b-33d5f925dce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:39.668274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1051d-6397-466c-beec-fae2687227e8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-69215bad-3ee0-4c72-a34b-33d5f925dce5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:39.677044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:39.677961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:39.678757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:39.679357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 213/417] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:39.692420 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69215bad-3ee0-4c72-a34b-33d5f925dce5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:39.736271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:39.861197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1051d-6397-466c-beec-fae2687227e8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:39.882270 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69215bad-3ee0-4c72-a34b-33d5f925dce5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:39.922507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b063e-7e3c-4304-bf96-38f4bd996987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:39.923254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b063e-7e3c-4304-bf96-38f4bd996987 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.923534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b063e-7e3c-4304-bf96-38f4bd996987 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.923806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b063e-7e3c-4304-bf96-38f4bd996987 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:39.924584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b063e-7e3c-4304-bf96-38f4bd996987 None admin] RBAC: Adding query filter params (name=member) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:39.924890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b063e-7e3c-4304-bf96-38f4bd996987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:39.926277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b063e-7e3c-4304-bf96-38f4bd996987 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:39.933293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d8d1b-eec0-4a4e-82ca-94cd9c2bad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:39.934194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d8d1b-eec0-4a4e-82ca-94cd9c2bad69 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.934473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d8d1b-eec0-4a4e-82ca-94cd9c2bad69 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.934743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d8d1b-eec0-4a4e-82ca-94cd9c2bad69 None admin] PATH_INFO: `/v3/projects/d34d18aa450b41d2ac1001da2d9a5476/groups/ff0446e46bde4cb892e4a5b48bebe14b/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:39.937904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d8d1b-eec0-4a4e-82ca-94cd9c2bad69 None admin] RBAC: Authorizing `identity:create_grant(project_id=d34d18aa450b41d2ac1001da2d9a5476, group_id=ff0446e46bde4cb892e4a5b48bebe14b, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:39.939358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d8d1b-eec0-4a4e-82ca-94cd9c2bad69 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:39.943591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 214/418] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:39.951734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b063e-7e3c-4304-bf96-38f4bd996987 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:39.971902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 205/419] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Jun 8 01:51:39 2022] PUT /identity/v3/projects/d34d18aa450b41d2ac1001da2d9a5476/groups/ff0446e46bde4cb892e4a5b48bebe14b/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:51:39.980774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f2097-9a3d-444e-b3e9-e06b6bcd8816 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.981081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f2097-9a3d-444e-b3e9-e06b6bcd8816 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.981328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f2097-9a3d-444e-b3e9-e06b6bcd8816 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:39.982937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 206/420] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:39.988614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736f25b-e200-4dff-8a23-ed9439b0a07f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:39.988934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736f25b-e200-4dff-8a23-ed9439b0a07f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:39.989205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736f25b-e200-4dff-8a23-ed9439b0a07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:40.026385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d736f25b-e200-4dff-8a23-ed9439b0a07f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:40.027631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d736f25b-e200-4dff-8a23-ed9439b0a07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:40.056482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:40.140613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:40.166460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b063e-7e3c-4304-bf96-38f4bd996987 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:40.199229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 207/421] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:40.213969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7bf56-8123-4846-84a5-437c568b74c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:40.214971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7bf56-8123-4846-84a5-437c568b74c9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:40.215448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7bf56-8123-4846-84a5-437c568b74c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:40.215933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7bf56-8123-4846-84a5-437c568b74c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:40.223243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736f25b-e200-4dff-8a23-ed9439b0a07f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:40.304179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7bf56-8123-4846-84a5-437c568b74c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:40.306605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7bf56-8123-4846-84a5-437c568b74c9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:40.373147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736f25b-e200-4dff-8a23-ed9439b0a07f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:40.399857 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:40.426470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b085898-500d-45f8-8ace-33f1c30956af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:40.427290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b085898-500d-45f8-8ace-33f1c30956af None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:40.427574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b085898-500d-45f8-8ace-33f1c30956af None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:40.427849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b085898-500d-45f8-8ace-33f1c30956af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:40.432189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 215/422] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:40.437782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7bf56-8123-4846-84a5-437c568b74c9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:40.452485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b085898-500d-45f8-8ace-33f1c30956af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0b085898-500d-45f8-8ace-33f1c30956af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:40.467776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:40.468822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:40.469605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 208/423] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:40.477477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b085898-500d-45f8-8ace-33f1c30956af None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:40.500980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:40.596007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7bf56-8123-4846-84a5-437c568b74c9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:40.641276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcbdd1-ecdd-4296-947d-86f2d49c6448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:40.642222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcbdd1-ecdd-4296-947d-86f2d49c6448 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:40.642511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcbdd1-ecdd-4296-947d-86f2d49c6448 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:40.642785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcbdd1-ecdd-4296-947d-86f2d49c6448 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:40.646720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcbdd1-ecdd-4296-947d-86f2d49c6448 None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:40.652127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcbdd1-ecdd-4296-947d-86f2d49c6448 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:40.654095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 216/424] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:40 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:40.659805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcbdd1-ecdd-4296-947d-86f2d49c6448 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:40.711421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b085898-500d-45f8-8ace-33f1c30956af None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:40.759280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b156387-0996-4d2c-8ea4-bef43965a3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:40.759965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b156387-0996-4d2c-8ea4-bef43965a3ba None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:40.760294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b156387-0996-4d2c-8ea4-bef43965a3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:40.760767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b156387-0996-4d2c-8ea4-bef43965a3ba None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:40.762205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b156387-0996-4d2c-8ea4-bef43965a3ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:40.762764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b156387-0996-4d2c-8ea4-bef43965a3ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:40.764268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:40.767097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b156387-0996-4d2c-8ea4-bef43965a3ba None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:40.778728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 209/425] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:40.784256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b156387-0996-4d2c-8ea4-bef43965a3ba None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:40.864810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:40.886323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcbdd1-ecdd-4296-947d-86f2d49c6448 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:40.898428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b156387-0996-4d2c-8ea4-bef43965a3ba None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:40.935780 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54fff1-a245-4da9-b6b2-55631c032020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:40.936745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54fff1-a245-4da9-b6b2-55631c032020 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:40.937206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54fff1-a245-4da9-b6b2-55631c032020 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:40.937656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54fff1-a245-4da9-b6b2-55631c032020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:40.971504 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000376ec-1f47-4d75-8ec0-d2e63a5f6c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:40.972515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000376ec-1f47-4d75-8ec0-d2e63a5f6c22 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:40.972886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000376ec-1f47-4d75-8ec0-d2e63a5f6c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:40.973235 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000376ec-1f47-4d75-8ec0-d2e63a5f6c22 None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:40.976107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000376ec-1f47-4d75-8ec0-d2e63a5f6c22 None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:40.977862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000376ec-1f47-4d75-8ec0-d2e63a5f6c22 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:40.980022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 217/426] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:40 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:40.985483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000376ec-1f47-4d75-8ec0-d2e63a5f6c22 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:41.014033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54fff1-a245-4da9-b6b2-55631c032020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:41.015878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54fff1-a245-4da9-b6b2-55631c032020 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:41.096836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:41.129803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 210/427] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 08 01:51:41.139282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54fff1-a245-4da9-b6b2-55631c032020 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:41.187003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000376ec-1f47-4d75-8ec0-d2e63a5f6c22 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:41.225025 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:41.248906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6b3bb-81c0-4ee6-9cec-b88e0f94f063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:41.249573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6b3bb-81c0-4ee6-9cec-b88e0f94f063 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:41.249857 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6b3bb-81c0-4ee6-9cec-b88e0f94f063 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:41.250133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6b3bb-81c0-4ee6-9cec-b88e0f94f063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:41.250974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6b3bb-81c0-4ee6-9cec-b88e0f94f063 None admin] RBAC: Adding query filter params (group.id=d4941b980ca24cca8e0c12ef50bce322, role.id=61490cf1d7334421b84ff67552b79bfc, scope.project.id=faa8ebf890584746af20166b89a7e162) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:41.251288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6b3bb-81c0-4ee6-9cec-b88e0f94f063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:41.252758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6b3bb-81c0-4ee6-9cec-b88e0f94f063 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:41.264276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 218/428] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jun 8 01:51:40 2022] GET /identity/v3/role_assignments?group.id=d4941b980ca24cca8e0c12ef50bce322&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=61490cf1d7334421b84ff67552b79bfc => generated 611 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:41.299726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54fff1-a245-4da9-b6b2-55631c032020 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:41.370721 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7b1e5-4b2a-4378-8773-3e4232d53629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:41.371612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7b1e5-4b2a-4378-8773-3e4232d53629 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:41.371889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7b1e5-4b2a-4378-8773-3e4232d53629 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:41.372157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7b1e5-4b2a-4378-8773-3e4232d53629 None admin] PATH_INFO: `/v3/projects/faa8ebf890584746af20166b89a7e162` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:41.373372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7b1e5-4b2a-4378-8773-3e4232d53629 None admin] RBAC: Authorizing `identity:get_project(project_id=faa8ebf890584746af20166b89a7e162)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:41.374840 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7b1e5-4b2a-4378-8773-3e4232d53629 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:41.376849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 211/429] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:41 2022] GET /identity/v3/projects/faa8ebf890584746af20166b89a7e162 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:41.382321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6b3bb-81c0-4ee6-9cec-b88e0f94f063 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:41.418369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1367e9-53b8-415c-8059-5e6661f6cc6b None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:41.419024 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1367e9-53b8-415c-8059-5e6661f6cc6b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:41.419308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1367e9-53b8-415c-8059-5e6661f6cc6b None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:41.421029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 212/430] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:41.426983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc891c-6725-4115-8ed1-4f2006e9bf33 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:41.427290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc891c-6725-4115-8ed1-4f2006e9bf33 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:41.427538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc891c-6725-4115-8ed1-4f2006e9bf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:41.449104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-cadc891c-6725-4115-8ed1-4f2006e9bf33 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:41.450671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cadc891c-6725-4115-8ed1-4f2006e9bf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:41.516828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:41.570529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6b3bb-81c0-4ee6-9cec-b88e0f94f063 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:41.583049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 213/431] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:41.608299 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbd6ff-f5c7-4877-92e8-88d6d478d7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:41.608951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbd6ff-f5c7-4877-92e8-88d6d478d7eb None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:41.609229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbd6ff-f5c7-4877-92e8-88d6d478d7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:41.609329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc891c-6725-4115-8ed1-4f2006e9bf33 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:41.609497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbd6ff-f5c7-4877-92e8-88d6d478d7eb None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:41.610676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbd6ff-f5c7-4877-92e8-88d6d478d7eb None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:41.612137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbd6ff-f5c7-4877-92e8-88d6d478d7eb None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:41.614064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 219/432] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:41 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:41.625534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbd6ff-f5c7-4877-92e8-88d6d478d7eb None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:41.709695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:41.751874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc891c-6725-4115-8ed1-4f2006e9bf33 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:41.784215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbd6ff-f5c7-4877-92e8-88d6d478d7eb None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:41.791447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc356e99-5d98-483b-91c7-ad9a82f763e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:41.792612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc356e99-5d98-483b-91c7-ad9a82f763e7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:41.793075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc356e99-5d98-483b-91c7-ad9a82f763e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:41.793573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc356e99-5d98-483b-91c7-ad9a82f763e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:41.805149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc356e99-5d98-483b-91c7-ad9a82f763e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-dc356e99-5d98-483b-91c7-ad9a82f763e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:41.814652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:41.815556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:41.816347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:41.818887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 214/433] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:41.825371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc356e99-5d98-483b-91c7-ad9a82f763e7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:41.825904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f781c8-a959-4407-9b81-aee557dca8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:41.831032 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f781c8-a959-4407-9b81-aee557dca8fa None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:41.831907 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f781c8-a959-4407-9b81-aee557dca8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:41.832332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f781c8-a959-4407-9b81-aee557dca8fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:41.833337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f781c8-a959-4407-9b81-aee557dca8fa None admin] RBAC: Adding query filter params (group.id=d4941b980ca24cca8e0c12ef50bce322, role.id=c45c9440d2b8447ab50b814c6be35bf1, scope.project.id=faa8ebf890584746af20166b89a7e162) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:41.833797 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f781c8-a959-4407-9b81-aee557dca8fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:41.835409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f781c8-a959-4407-9b81-aee557dca8fa None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:41.844630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 220/434] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jun 8 01:51:41 2022] GET /identity/v3/role_assignments?group.id=d4941b980ca24cca8e0c12ef50bce322&scope.project.id=faa8ebf890584746af20166b89a7e162&role.id=c45c9440d2b8447ab50b814c6be35bf1 => generated 611 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:41.863224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:41.965422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc356e99-5d98-483b-91c7-ad9a82f763e7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:42.005494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f681a6-3f29-4d0e-bf6a-6a340c4912d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:42.006184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f681a6-3f29-4d0e-bf6a-6a340c4912d7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.006463 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f681a6-3f29-4d0e-bf6a-6a340c4912d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.006736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f681a6-3f29-4d0e-bf6a-6a340c4912d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.007503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f681a6-3f29-4d0e-bf6a-6a340c4912d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:42.007806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f681a6-3f29-4d0e-bf6a-6a340c4912d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:42.009776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f681a6-3f29-4d0e-bf6a-6a340c4912d7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:42.023776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 215/435] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:42.025308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f781c8-a959-4407-9b81-aee557dca8fa None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:42.144253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f781c8-a959-4407-9b81-aee557dca8fa None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:42.164079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:42.178705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65669486-8dbf-4fda-9d6a-bbecc6776369 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.178994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65669486-8dbf-4fda-9d6a-bbecc6776369 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.179269 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65669486-8dbf-4fda-9d6a-bbecc6776369 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.182113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 216/436] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:42.187392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b6557-2c63-4205-b3b7-ce366e9e697d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.187678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b6557-2c63-4205-b3b7-ce366e9e697d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.187956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b6557-2c63-4205-b3b7-ce366e9e697d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.188589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ad904-a028-4023-9a1e-42c11ee6163c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:42.189482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ad904-a028-4023-9a1e-42c11ee6163c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.190162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ad904-a028-4023-9a1e-42c11ee6163c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.190643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ad904-a028-4023-9a1e-42c11ee6163c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.212500 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-642b6557-2c63-4205-b3b7-ce366e9e697d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:42.213714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-642b6557-2c63-4205-b3b7-ce366e9e697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:42.271878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ad904-a028-4023-9a1e-42c11ee6163c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:42.273372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ad904-a028-4023-9a1e-42c11ee6163c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:42.311140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 217/437] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:42.337968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b6557-2c63-4205-b3b7-ce366e9e697d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:42.366199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:42.411536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 221/438] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 08 01:51:42.417552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ad904-a028-4023-9a1e-42c11ee6163c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:42.440553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:42.466450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b6557-2c63-4205-b3b7-ce366e9e697d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:42.503583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd72d03-655f-454f-9344-d4aab3d443da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:42.504386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd72d03-655f-454f-9344-d4aab3d443da None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.504695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd72d03-655f-454f-9344-d4aab3d443da None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.504968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd72d03-655f-454f-9344-d4aab3d443da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.515418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd72d03-655f-454f-9344-d4aab3d443da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-6dd72d03-655f-454f-9344-d4aab3d443da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:42.522955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:42.523852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:42.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 218/439] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:51:42.530620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd72d03-655f-454f-9344-d4aab3d443da None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:42.550395 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ad904-a028-4023-9a1e-42c11ee6163c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:42.588953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc49fe5-48b5-42fa-8f6a-066595069646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:42.591192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc49fe5-48b5-42fa-8f6a-066595069646 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.591934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc49fe5-48b5-42fa-8f6a-066595069646 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.592460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc49fe5-48b5-42fa-8f6a-066595069646 None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.594479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc49fe5-48b5-42fa-8f6a-066595069646 None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:42.596646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc49fe5-48b5-42fa-8f6a-066595069646 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:42.604061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 222/440] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:42 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:42.605081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc49fe5-48b5-42fa-8f6a-066595069646 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:42.617100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:42.639569 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd72d03-655f-454f-9344-d4aab3d443da None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:42.674504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bca62-6fc7-4362-98cb-4eb8cb8b2295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:42.675107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bca62-6fc7-4362-98cb-4eb8cb8b2295 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.675393 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bca62-6fc7-4362-98cb-4eb8cb8b2295 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.675673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bca62-6fc7-4362-98cb-4eb8cb8b2295 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.676438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bca62-6fc7-4362-98cb-4eb8cb8b2295 None admin] RBAC: Adding query filter params (name=member) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:42.676751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bca62-6fc7-4362-98cb-4eb8cb8b2295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:42.678148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bca62-6fc7-4362-98cb-4eb8cb8b2295 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:42.686908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 219/441] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:51:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:42.689634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452f07d-57e9-4ac9-bbf2-e501c00ac552 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.689922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452f07d-57e9-4ac9-bbf2-e501c00ac552 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.690199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452f07d-57e9-4ac9-bbf2-e501c00ac552 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.691762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 220/442] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:42.693217 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452f07d-57e9-4ac9-bbf2-e501c00ac552 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:42.697416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:42.713089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc49fe5-48b5-42fa-8f6a-066595069646 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:42.747014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fe236-0c23-4d94-99fc-8d7d24591d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:42.747872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe236-0c23-4d94-99fc-8d7d24591d23 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.748160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe236-0c23-4d94-99fc-8d7d24591d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.748465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe236-0c23-4d94-99fc-8d7d24591d23 None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.749642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fe236-0c23-4d94-99fc-8d7d24591d23 None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:42.751085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fe236-0c23-4d94-99fc-8d7d24591d23 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:42.752885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 223/443] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:42 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:42.753864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40f17a-9d7a-40b7-a083-22d5e0d0cec1 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.754151 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40f17a-9d7a-40b7-a083-22d5e0d0cec1 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.754431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40f17a-9d7a-40b7-a083-22d5e0d0cec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.770791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-9d40f17a-9d7a-40b7-a083-22d5e0d0cec1 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:42.771984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-9d40f17a-9d7a-40b7-a083-22d5e0d0cec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:42.798708 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452f07d-57e9-4ac9-bbf2-e501c00ac552 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:42.832243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db3eac-dd9b-4736-a34f-c73e94c7d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:42.832927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db3eac-dd9b-4736-a34f-c73e94c7d95d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.833214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db3eac-dd9b-4736-a34f-c73e94c7d95d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.833490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db3eac-dd9b-4736-a34f-c73e94c7d95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.838430 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 224/444] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:42.839248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40f17a-9d7a-40b7-a083-22d5e0d0cec1 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:42.845452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:42.903183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db3eac-dd9b-4736-a34f-c73e94c7d95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:42.904671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db3eac-dd9b-4736-a34f-c73e94c7d95d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:42.928777 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:42.947815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40f17a-9d7a-40b7-a083-22d5e0d0cec1 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:42.981296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cccc1-6977-4d23-9b75-df25a5be6d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:42.981943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cccc1-6977-4d23-9b75-df25a5be6d75 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:42.982220 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cccc1-6977-4d23-9b75-df25a5be6d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:42.982498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cccc1-6977-4d23-9b75-df25a5be6d75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:42.983313 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cccc1-6977-4d23-9b75-df25a5be6d75 None admin] RBAC: Adding query filter params (group.id=d4941b980ca24cca8e0c12ef50bce322, role.id=c45c9440d2b8447ab50b814c6be35bf1, scope.project.id=defbc73a230d43b5afdb17c929c4e61a) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:42.983622 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cccc1-6977-4d23-9b75-df25a5be6d75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:42.985102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cccc1-6977-4d23-9b75-df25a5be6d75 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:42.990988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 225/445] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jun 8 01:51:42 2022] GET /identity/v3/role_assignments?group.id=d4941b980ca24cca8e0c12ef50bce322&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=c45c9440d2b8447ab50b814c6be35bf1 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:42.991663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cccc1-6977-4d23-9b75-df25a5be6d75 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:43.012436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 221/446] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:43.017108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db3eac-dd9b-4736-a34f-c73e94c7d95d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:43.068294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:43.110566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cccc1-6977-4d23-9b75-df25a5be6d75 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:43.136350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db3eac-dd9b-4736-a34f-c73e94c7d95d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:43.147787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad54800-3a6e-49eb-84ef-260feb8fe759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:43.148620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad54800-3a6e-49eb-84ef-260feb8fe759 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:43.148907 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad54800-3a6e-49eb-84ef-260feb8fe759 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:43.149718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad54800-3a6e-49eb-84ef-260feb8fe759 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:43.161699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad54800-3a6e-49eb-84ef-260feb8fe759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-dad54800-3a6e-49eb-84ef-260feb8fe759 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:43.170365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:43.171345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:43.172593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 226/447] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:51:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:43.177416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad54800-3a6e-49eb-84ef-260feb8fe759 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:43.179208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfcf39-10d4-4882-950d-d2e2e8e938ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:43.180481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfcf39-10d4-4882-950d-d2e2e8e938ba None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:43.180951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfcf39-10d4-4882-950d-d2e2e8e938ba None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:43.181227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfcf39-10d4-4882-950d-d2e2e8e938ba None admin] PATH_INFO: `/v3/projects/defbc73a230d43b5afdb17c929c4e61a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:43.182671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfcf39-10d4-4882-950d-d2e2e8e938ba None admin] RBAC: Authorizing `identity:get_project(project_id=defbc73a230d43b5afdb17c929c4e61a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:43.184130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfcf39-10d4-4882-950d-d2e2e8e938ba None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:43.185878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 222/448] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:43 2022] GET /identity/v3/projects/defbc73a230d43b5afdb17c929c4e61a => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:43.187439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:43.193485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfcf39-10d4-4882-950d-d2e2e8e938ba None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:43.290416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad54800-3a6e-49eb-84ef-260feb8fe759 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:43.302400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfcf39-10d4-4882-950d-d2e2e8e938ba None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:43.312838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:43.323490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce866b5d-879a-428a-be34-1bb190b619c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:43.324161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce866b5d-879a-428a-be34-1bb190b619c8 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:43.324481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce866b5d-879a-428a-be34-1bb190b619c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:43.324768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce866b5d-879a-428a-be34-1bb190b619c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:43.325515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce866b5d-879a-428a-be34-1bb190b619c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:43.325819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce866b5d-879a-428a-be34-1bb190b619c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:43.327196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce866b5d-879a-428a-be34-1bb190b619c8 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:43.335306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 227/449] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:43.336365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd302f-2524-4002-bcef-2c376ae12f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:43.337810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd302f-2524-4002-bcef-2c376ae12f6a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:43.338090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd302f-2524-4002-bcef-2c376ae12f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:43.338360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd302f-2524-4002-bcef-2c376ae12f6a None admin] PATH_INFO: `/v3/groups/d4941b980ca24cca8e0c12ef50bce322` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:43.339528 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd302f-2524-4002-bcef-2c376ae12f6a None admin] RBAC: Authorizing `identity:get_group(group_id=d4941b980ca24cca8e0c12ef50bce322)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:43.341036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cd302f-2524-4002-bcef-2c376ae12f6a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:43.341462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce866b5d-879a-428a-be34-1bb190b619c8 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:43.347102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 223/450] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:43 2022] GET /identity/v3/groups/d4941b980ca24cca8e0c12ef50bce322 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:43.348055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd302f-2524-4002-bcef-2c376ae12f6a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:43.405953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:43.444228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce866b5d-879a-428a-be34-1bb190b619c8 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:43.448012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cd302f-2524-4002-bcef-2c376ae12f6a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:43.478751 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa315537-4a97-43a6-a112-05fb495ea7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:43.479404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa315537-4a97-43a6-a112-05fb495ea7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:43.479688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa315537-4a97-43a6-a112-05fb495ea7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:43.479962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa315537-4a97-43a6-a112-05fb495ea7b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:43.481870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705ed79-2d76-48d8-9f9b-7d26840fbc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:43.482514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705ed79-2d76-48d8-9f9b-7d26840fbc61 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:43.482791 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705ed79-2d76-48d8-9f9b-7d26840fbc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:43.483072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705ed79-2d76-48d8-9f9b-7d26840fbc61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:43.483885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705ed79-2d76-48d8-9f9b-7d26840fbc61 None admin] RBAC: Adding query filter params (group.id=d4941b980ca24cca8e0c12ef50bce322, role.id=61490cf1d7334421b84ff67552b79bfc, scope.project.id=defbc73a230d43b5afdb17c929c4e61a) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:43.484191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705ed79-2d76-48d8-9f9b-7d26840fbc61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:43.485658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705ed79-2d76-48d8-9f9b-7d26840fbc61 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:43.491356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 224/451] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jun 8 01:51:43 2022] GET /identity/v3/role_assignments?group.id=d4941b980ca24cca8e0c12ef50bce322&scope.project.id=defbc73a230d43b5afdb17c929c4e61a&role.id=61490cf1d7334421b84ff67552b79bfc => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:43.552165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa315537-4a97-43a6-a112-05fb495ea7b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:43.553699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa315537-4a97-43a6-a112-05fb495ea7b2 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:43.628355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:43.667792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 228/452] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 08 01:51:43.673446 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705ed79-2d76-48d8-9f9b-7d26840fbc61 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:43.718214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:43.777484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705ed79-2d76-48d8-9f9b-7d26840fbc61 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:43.812420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a3c04-4336-4d95-8be5-a2b027bbf935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:43.813304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a3c04-4336-4d95-8be5-a2b027bbf935 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:43.813590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a3c04-4336-4d95-8be5-a2b027bbf935 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:43.813870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a3c04-4336-4d95-8be5-a2b027bbf935 None admin] PATH_INFO: `/v3/projects/d34d18aa450b41d2ac1001da2d9a5476` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:43.815056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a3c04-4336-4d95-8be5-a2b027bbf935 None admin] RBAC: Authorizing `identity:get_project(project_id=d34d18aa450b41d2ac1001da2d9a5476)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:43.816531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a3c04-4336-4d95-8be5-a2b027bbf935 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:43.819047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 225/453] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:51:43 2022] GET /identity/v3/projects/d34d18aa450b41d2ac1001da2d9a5476 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:43.824886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa315537-4a97-43a6-a112-05fb495ea7b2 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:43.930862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa315537-4a97-43a6-a112-05fb495ea7b2 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:43.965241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e04eb3-1680-4c4e-a377-8c63855e810f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:43.966104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e04eb3-1680-4c4e-a377-8c63855e810f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:43.967269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e04eb3-1680-4c4e-a377-8c63855e810f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:43.967269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e04eb3-1680-4c4e-a377-8c63855e810f None admin] PATH_INFO: `/v3/groups/ff0446e46bde4cb892e4a5b48bebe14b` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:43.967818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e04eb3-1680-4c4e-a377-8c63855e810f None admin] RBAC: Authorizing `identity:get_group(group_id=ff0446e46bde4cb892e4a5b48bebe14b)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:43.969346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e04eb3-1680-4c4e-a377-8c63855e810f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:43.970897 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 229/454] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Jun 8 01:51:43 2022] GET /identity/v3/groups/ff0446e46bde4cb892e4a5b48bebe14b => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:43.976818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a3c04-4336-4d95-8be5-a2b027bbf935 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:44.079056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a3c04-4336-4d95-8be5-a2b027bbf935 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:44.089330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:44.112914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d085b0-9000-4fe4-9b99-5a7e7b0ed03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:44.113570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d085b0-9000-4fe4-9b99-5a7e7b0ed03e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:44.113856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d085b0-9000-4fe4-9b99-5a7e7b0ed03e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:44.114133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d085b0-9000-4fe4-9b99-5a7e7b0ed03e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:44.114962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d085b0-9000-4fe4-9b99-5a7e7b0ed03e None admin] RBAC: Adding query filter params (group.id=ff0446e46bde4cb892e4a5b48bebe14b, role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=d34d18aa450b41d2ac1001da2d9a5476) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:44.115281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d085b0-9000-4fe4-9b99-5a7e7b0ed03e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:44.116754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d085b0-9000-4fe4-9b99-5a7e7b0ed03e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:44.123022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 226/455] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Jun 8 01:51:43 2022] GET /identity/v3/role_assignments?group.id=ff0446e46bde4cb892e4a5b48bebe14b&scope.project.id=d34d18aa450b41d2ac1001da2d9a5476&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:46.953767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffae656-fe3d-435f-8285-a5d5984ac436 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:46.954728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffae656-fe3d-435f-8285-a5d5984ac436 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:46.955210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffae656-fe3d-435f-8285-a5d5984ac436 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:46.956822 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 230/456] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:46.970151 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ba4a7-3c32-49b9-8605-6aa0154b965f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:46.970613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ba4a7-3c32-49b9-8605-6aa0154b965f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:46.970711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ba4a7-3c32-49b9-8605-6aa0154b965f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:46.993868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-632ba4a7-3c32-49b9-8605-6aa0154b965f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:46.995317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-632ba4a7-3c32-49b9-8605-6aa0154b965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:47.090705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 227/457] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:47.092256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e554dfd-97df-4b80-801b-a31e4ec9190f None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.092651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e554dfd-97df-4b80-801b-a31e4ec9190f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.093007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e554dfd-97df-4b80-801b-a31e4ec9190f None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.094631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 231/458] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:47.099896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86db0e2-e3a5-43fd-976b-1c60f0d03abf None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.100195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86db0e2-e3a5-43fd-976b-1c60f0d03abf None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.100507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86db0e2-e3a5-43fd-976b-1c60f0d03abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.123402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d86db0e2-e3a5-43fd-976b-1c60f0d03abf None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:47.124673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d86db0e2-e3a5-43fd-976b-1c60f0d03abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:47.136371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e554dfd-97df-4b80-801b-a31e4ec9190f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:47.195283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:47.210915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 228/459] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:47.213234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dfaec-952e-425e-87e2-1b932bead5de None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.213526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dfaec-952e-425e-87e2-1b932bead5de None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.213806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dfaec-952e-425e-87e2-1b932bead5de None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.215159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 229/460] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:47.223017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf7f41-faaa-4d14-870e-d699c58c083c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.223504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf7f41-faaa-4d14-870e-d699c58c083c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.223994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf7f41-faaa-4d14-870e-d699c58c083c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.252362 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e3bf7f41-faaa-4d14-870e-d699c58c083c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:47.253604 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e3bf7f41-faaa-4d14-870e-d699c58c083c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:47.272226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e554dfd-97df-4b80-801b-a31e4ec9190f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:47.289614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:47.312931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d36773-4182-4926-8c76-b2fbc9c2eeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:47.314113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d36773-4182-4926-8c76-b2fbc9c2eeff None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.314658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d36773-4182-4926-8c76-b2fbc9c2eeff None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.315140 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d36773-4182-4926-8c76-b2fbc9c2eeff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.326683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d36773-4182-4926-8c76-b2fbc9c2eeff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:47.328475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d36773-4182-4926-8c76-b2fbc9c2eeff None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:47.342098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 230/461] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:47.343055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf7f41-faaa-4d14-870e-d699c58c083c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-f0d36773-4182-4926-8c76-b2fbc9c2eeff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:47.347945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:47.348985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:47.349831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:47.349831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:47.349831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:47.349831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:47.351604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 232/462] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:47.361357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d36773-4182-4926-8c76-b2fbc9c2eeff None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:47.464264 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf7f41-faaa-4d14-870e-d699c58c083c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:47.491525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:47.503067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f44b8-cdce-4351-a779-f2bc7119ac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:47.503972 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f44b8-cdce-4351-a779-f2bc7119ac41 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.504357 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f44b8-cdce-4351-a779-f2bc7119ac41 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.504679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f44b8-cdce-4351-a779-f2bc7119ac41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.507441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d36773-4182-4926-8c76-b2fbc9c2eeff None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:47.515650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f44b8-cdce-4351-a779-f2bc7119ac41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:47.517168 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f44b8-cdce-4351-a779-f2bc7119ac41 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-718f44b8-cdce-4351-a779-f2bc7119ac41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:47.535498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:47.542610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:47.543468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:47.543468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:47.543468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:47.543468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:47.543468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 231/463] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:47.543468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f44b8-cdce-4351-a779-f2bc7119ac41 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:47.551883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077817e1-b290-48dd-a634-4624859658a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:47.552862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077817e1-b290-48dd-a634-4624859658a2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.553148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077817e1-b290-48dd-a634-4624859658a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.553422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077817e1-b290-48dd-a634-4624859658a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.554416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077817e1-b290-48dd-a634-4624859658a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:47.555344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077817e1-b290-48dd-a634-4624859658a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:47.557079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077817e1-b290-48dd-a634-4624859658a2 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:47.557482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:47.587402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 233/464] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:47.587402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1147dab-a7a4-424d-ab65-1c31a7a8b3b4 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.587402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1147dab-a7a4-424d-ab65-1c31a7a8b3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.587402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1147dab-a7a4-424d-ab65-1c31a7a8b3b4 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.588311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 234/465] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:47.593915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1147dab-a7a4-424d-ab65-1c31a7a8b3b4 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:47.643979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f44b8-cdce-4351-a779-f2bc7119ac41 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:47.679603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a903bf8-1dbb-440a-86f0-ab00769a8af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:47.680593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a903bf8-1dbb-440a-86f0-ab00769a8af7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.680888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a903bf8-1dbb-440a-86f0-ab00769a8af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.681157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a903bf8-1dbb-440a-86f0-ab00769a8af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.692520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a903bf8-1dbb-440a-86f0-ab00769a8af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:47.693978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a903bf8-1dbb-440a-86f0-ab00769a8af7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2a903bf8-1dbb-440a-86f0-ab00769a8af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:47.712559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:47.713550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:47.714457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:47.714457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:47.714457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:47.714457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:47.714678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1147dab-a7a4-424d-ab65-1c31a7a8b3b4 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:47.715301 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 232/466] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:47.715301 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a903bf8-1dbb-440a-86f0-ab00769a8af7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:47.754263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6f82b-e0b3-448e-b4c2-96b0f5b9102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:47.755051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6f82b-e0b3-448e-b4c2-96b0f5b9102b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.755333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6f82b-e0b3-448e-b4c2-96b0f5b9102b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.755609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6f82b-e0b3-448e-b4c2-96b0f5b9102b None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.756328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6f82b-e0b3-448e-b4c2-96b0f5b9102b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:47.756781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6f82b-e0b3-448e-b4c2-96b0f5b9102b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:47.758246 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6f82b-e0b3-448e-b4c2-96b0f5b9102b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:47.774835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 235/467] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:47.774835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f138cc-c0dd-4f2d-af0b-44ebc0b11c1d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.774835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f138cc-c0dd-4f2d-af0b-44ebc0b11c1d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.774835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f138cc-c0dd-4f2d-af0b-44ebc0b11c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.787637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:47.790321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-09f138cc-c0dd-4f2d-af0b-44ebc0b11c1d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:47.791637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-09f138cc-c0dd-4f2d-af0b-44ebc0b11c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:47.793724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:47.826427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a903bf8-1dbb-440a-86f0-ab00769a8af7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:47.861728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 236/468] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:47.862104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a284fa8-f7be-41b0-9e69-db3e257e2588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:47.862790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a284fa8-f7be-41b0-9e69-db3e257e2588 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:47.863076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a284fa8-f7be-41b0-9e69-db3e257e2588 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:47.863197 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f138cc-c0dd-4f2d-af0b-44ebc0b11c1d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:47.863351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a284fa8-f7be-41b0-9e69-db3e257e2588 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:47.864057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a284fa8-f7be-41b0-9e69-db3e257e2588 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:47.865529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a284fa8-f7be-41b0-9e69-db3e257e2588 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:47.888472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 233/469] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:51:47 2022] POST /identity/v3/users => generated 246 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:51:47.893466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a284fa8-f7be-41b0-9e69-db3e257e2588 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:48.013556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f138cc-c0dd-4f2d-af0b-44ebc0b11c1d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:48.036823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a284fa8-f7be-41b0-9e69-db3e257e2588 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:48.053391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0cd33-c800-4fbd-a62c-6291d887742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:48.054536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0cd33-c800-4fbd-a62c-6291d887742d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:48.054536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0cd33-c800-4fbd-a62c-6291d887742d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:48.055007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0cd33-c800-4fbd-a62c-6291d887742d None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:48.056048 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0cd33-c800-4fbd-a62c-6291d887742d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:48.056611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0cd33-c800-4fbd-a62c-6291d887742d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:48.058330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0cd33-c800-4fbd-a62c-6291d887742d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:48.066172 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:48.068151 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:48.078944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 237/470] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:48.078944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0cd33-c800-4fbd-a62c-6291d887742d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:48.094254 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd20b1-2ee3-4e16-807f-6483e14caa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:48.095407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd20b1-2ee3-4e16-807f-6483e14caa19 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:48.095887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd20b1-2ee3-4e16-807f-6483e14caa19 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:48.096308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd20b1-2ee3-4e16-807f-6483e14caa19 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:48.097318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd20b1-2ee3-4e16-807f-6483e14caa19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:48.099024 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd20b1-2ee3-4e16-807f-6483e14caa19 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:48.123282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 234/471] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:51:47 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:51:48.124311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd20b1-2ee3-4e16-807f-6483e14caa19 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:48.199488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0cd33-c800-4fbd-a62c-6291d887742d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:48.240255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaf0ce-3d26-4baf-af06-34554fa327d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:48.241324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaf0ce-3d26-4baf-af06-34554fa327d4 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:48.241577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaf0ce-3d26-4baf-af06-34554fa327d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:48.241938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaf0ce-3d26-4baf-af06-34554fa327d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:48.247011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd20b1-2ee3-4e16-807f-6483e14caa19 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:48.257790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaf0ce-3d26-4baf-af06-34554fa327d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:48.259869 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaf0ce-3d26-4baf-af06-34554fa327d4 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-cedaf0ce-3d26-4baf-af06-34554fa327d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:48.280008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:48.280915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:48.281771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:48.281771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:48.281771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:48.281771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:48.286503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 238/472] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:48.288012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaf0ce-3d26-4baf-af06-34554fa327d4 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:48.292790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654327e-cbe9-4f23-bdc3-0bb6f1cc93cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:48.293739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654327e-cbe9-4f23-bdc3-0bb6f1cc93cd None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:48.294204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654327e-cbe9-4f23-bdc3-0bb6f1cc93cd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:48.294737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654327e-cbe9-4f23-bdc3-0bb6f1cc93cd None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:48.295650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654327e-cbe9-4f23-bdc3-0bb6f1cc93cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:48.297417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654327e-cbe9-4f23-bdc3-0bb6f1cc93cd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:48.327187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 235/473] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:51:48 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:51:48.352824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:48.414240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaf0ce-3d26-4baf-af06-34554fa327d4 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:48.446547 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e5973-1c42-4bea-a61d-de8288c48625 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:48.446917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e5973-1c42-4bea-a61d-de8288c48625 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:48.447112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e5973-1c42-4bea-a61d-de8288c48625 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:48.448829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 236/474] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:51:48.454535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e6f0e-a3d7-4e24-9756-c6e4ffa8a7ff None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:48.454843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e6f0e-a3d7-4e24-9756-c6e4ffa8a7ff None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:48.455115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e6f0e-a3d7-4e24-9756-c6e4ffa8a7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:48.455799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993f978-29cf-4724-9b5c-6573161678fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:48.456602 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993f978-29cf-4724-9b5c-6573161678fb None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:48.456899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993f978-29cf-4724-9b5c-6573161678fb None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:48.457182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993f978-29cf-4724-9b5c-6573161678fb None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:48.457931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993f978-29cf-4724-9b5c-6573161678fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:48.458244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993f978-29cf-4724-9b5c-6573161678fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:48.459732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993f978-29cf-4724-9b5c-6573161678fb None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:48.478502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-939e6f0e-a3d7-4e24-9756-c6e4ffa8a7ff None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:48.478720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 239/475] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:48.479747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-939e6f0e-a3d7-4e24-9756-c6e4ffa8a7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:48.479931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993f978-29cf-4724-9b5c-6573161678fb None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:48.532877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:48.598415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 237/476] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:48.611955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993f978-29cf-4724-9b5c-6573161678fb None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:48.624950 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:48.637092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e6f0e-a3d7-4e24-9756-c6e4ffa8a7ff None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:48.652370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb71e0-fcc7-4e15-b86b-a6c1a0906d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:48.653124 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb71e0-fcc7-4e15-b86b-a6c1a0906d58 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:48.653411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb71e0-fcc7-4e15-b86b-a6c1a0906d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:48.653688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb71e0-fcc7-4e15-b86b-a6c1a0906d58 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:48.654412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb71e0-fcc7-4e15-b86b-a6c1a0906d58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:48.655985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb71e0-fcc7-4e15-b86b-a6c1a0906d58 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:48.683527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 240/477] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:51:48 2022] POST /identity/v3/users => generated 249 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:51:48.761655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e6f0e-a3d7-4e24-9756-c6e4ffa8a7ff None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:48.801946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca3ee3-047f-4f71-8a3b-7fda2be62c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:48.802539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca3ee3-047f-4f71-8a3b-7fda2be62c10 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:48.802818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca3ee3-047f-4f71-8a3b-7fda2be62c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:48.803152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca3ee3-047f-4f71-8a3b-7fda2be62c10 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:48.803902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca3ee3-047f-4f71-8a3b-7fda2be62c10 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:48.805440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca3ee3-047f-4f71-8a3b-7fda2be62c10 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:48.812196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.sql.core [None req-f0ca3ee3-047f-4f71-8a3b-7fda2be62c10 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 08 01:51:48.812196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 08 01:51:48.812196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [parameters: {'id': 'a30488b116014446ae62ce1446b67d52', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 08 01:51:48.812196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80425) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f0ca3ee3-047f-4f71-8a3b-7fda2be62c10 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = self._query(query) Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application conn.query(q) Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result.read() Jun 08 01:51:48.830331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return ref.to_dict() Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 08 01:51:48.831259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application next(self.gen) Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application yield resource Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application next(self.gen) Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.session.rollback() Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.force_reraise() Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise self.value Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application session.commit() Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 08 01:51:48.832905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._prepare_impl() Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.session.flush() Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._flush(objects) Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application compat.raise_( Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application flush_context.execute() Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rec.execute(self) Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 08 01:51:48.835062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 08 01:51:48.840580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = self._query(query) Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application conn.query(q) Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result.read() Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 08 01:51:48.841415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [parameters: {'id': 'a30488b116014446ae62ce1446b67d52', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:48.842220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 08 01:51:48.843041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:48.843870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 238/478] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Jun 8 01:51:48 2022] POST /identity/v3/roles => generated 144 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:48.843939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb71e0-fcc7-4e15-b86b-a6c1a0906d58 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:48.986928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb71e0-fcc7-4e15-b86b-a6c1a0906d58 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:49.023091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff71765-17d9-4df9-8b9a-b1b597fdaa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:49.024483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff71765-17d9-4df9-8b9a-b1b597fdaa02 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:49.024774 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff71765-17d9-4df9-8b9a-b1b597fdaa02 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:49.025055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff71765-17d9-4df9-8b9a-b1b597fdaa02 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:49.035849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff71765-17d9-4df9-8b9a-b1b597fdaa02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-fff71765-17d9-4df9-8b9a-b1b597fdaa02 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:49.044266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:49.046267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:49.047525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 241/479] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:51:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:49.054928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca3ee3-047f-4f71-8a3b-7fda2be62c10 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:49.092171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:49.170541 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca3ee3-047f-4f71-8a3b-7fda2be62c10 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:49.208490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d81ea-30c8-4913-bac5-eb3551712e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:49.209507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d81ea-30c8-4913-bac5-eb3551712e46 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:49.210024 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d81ea-30c8-4913-bac5-eb3551712e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:49.210511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d81ea-30c8-4913-bac5-eb3551712e46 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:49.211979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d81ea-30c8-4913-bac5-eb3551712e46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:49.212540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d81ea-30c8-4913-bac5-eb3551712e46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:49.214144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d81ea-30c8-4913-bac5-eb3551712e46 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:49.223731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 239/480] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:50.784810 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a33d77-ddf4-456a-89c9-79ab9e73dd04 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:50.785354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a33d77-ddf4-456a-89c9-79ab9e73dd04 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:50.785354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a33d77-ddf4-456a-89c9-79ab9e73dd04 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:50.787263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 242/481] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:50.792887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9d9e-0e18-4124-92e2-609238eefa5c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:50.793296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9d9e-0e18-4124-92e2-609238eefa5c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:50.793462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b9d9e-0e18-4124-92e2-609238eefa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:50.816130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1b2b9d9e-0e18-4124-92e2-609238eefa5c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:50.817354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1b2b9d9e-0e18-4124-92e2-609238eefa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:50.918695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 240/482] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:50.931749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78b7af-25da-4dcf-9458-9c141bcd8dec None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:50.932082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78b7af-25da-4dcf-9458-9c141bcd8dec None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:50.932319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78b7af-25da-4dcf-9458-9c141bcd8dec None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:50.933938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 243/483] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:50.939477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57361fcb-2d3a-4a12-9a39-4601130e5015 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:50.939770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57361fcb-2d3a-4a12-9a39-4601130e5015 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:50.940052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57361fcb-2d3a-4a12-9a39-4601130e5015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:50.951993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78b7af-25da-4dcf-9458-9c141bcd8dec None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:50.972083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-57361fcb-2d3a-4a12-9a39-4601130e5015 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:50.973321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-57361fcb-2d3a-4a12-9a39-4601130e5015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:51.001087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:51.002286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:51.089962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 241/484] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:51.110120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f156d2-1351-436a-ac9d-a4990673b43d None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.110412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f156d2-1351-436a-ac9d-a4990673b43d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.110681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f156d2-1351-436a-ac9d-a4990673b43d None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.112260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 242/485] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:51.117358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77cb42-3d2f-44f9-b48b-65c23554bf1d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.117749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77cb42-3d2f-44f9-b48b-65c23554bf1d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.118033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77cb42-3d2f-44f9-b48b-65c23554bf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.122381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78b7af-25da-4dcf-9458-9c141bcd8dec None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:51.137427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5e77cb42-3d2f-44f9-b48b-65c23554bf1d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:51.138610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5e77cb42-3d2f-44f9-b48b-65c23554bf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:51.159518 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83a547-49aa-41ca-b2e4-caa7c020e49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:51.160331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83a547-49aa-41ca-b2e4-caa7c020e49b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.160642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83a547-49aa-41ca-b2e4-caa7c020e49b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.161172 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83a547-49aa-41ca-b2e4-caa7c020e49b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.172662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83a547-49aa-41ca-b2e4-caa7c020e49b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-3c83a547-49aa-41ca-b2e4-caa7c020e49b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:51.180863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:51.181854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:51.182638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:51.186947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 244/486] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:51:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:51.187811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83a547-49aa-41ca-b2e4-caa7c020e49b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:51.224488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 243/487] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:51.225424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77cb42-3d2f-44f9-b48b-65c23554bf1d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:51.313237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:51.332780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83a547-49aa-41ca-b2e4-caa7c020e49b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:51.348818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:51.361104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77cb42-3d2f-44f9-b48b-65c23554bf1d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:51.375309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b7b76-0bfb-471c-8c7b-4b03ee23c78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:51.376126 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7b76-0bfb-471c-8c7b-4b03ee23c78e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.376655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7b76-0bfb-471c-8c7b-4b03ee23c78e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.376958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7b76-0bfb-471c-8c7b-4b03ee23c78e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.392439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b7b76-0bfb-471c-8c7b-4b03ee23c78e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-559b7b76-0bfb-471c-8c7b-4b03ee23c78e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:51.404181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:51.406293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:51.408148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:51.409618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f952b2-cc67-454b-a31d-27452b7f03ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:51.409618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f952b2-cc67-454b-a31d-27452b7f03ca None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.409618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f952b2-cc67-454b-a31d-27452b7f03ca None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.409618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f952b2-cc67-454b-a31d-27452b7f03ca None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.410134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f952b2-cc67-454b-a31d-27452b7f03ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:51.410134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f952b2-cc67-454b-a31d-27452b7f03ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:51.410134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f952b2-cc67-454b-a31d-27452b7f03ca None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:51.410524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 245/488] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:51:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:51.420419 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b7b76-0bfb-471c-8c7b-4b03ee23c78e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:51.422734 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 244/489] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:51:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:51.423462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f952b2-cc67-454b-a31d-27452b7f03ca None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:51.531991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b7b76-0bfb-471c-8c7b-4b03ee23c78e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:51.570535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7337cc9-f962-4281-a1e4-172009415cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:51.572498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7337cc9-f962-4281-a1e4-172009415cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.572933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7337cc9-f962-4281-a1e4-172009415cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.573214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7337cc9-f962-4281-a1e4-172009415cf3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.588223 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7337cc9-f962-4281-a1e4-172009415cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c7337cc9-f962-4281-a1e4-172009415cf3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:51.596361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:51.598803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 246/490] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:51:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:51.600742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7337cc9-f962-4281-a1e4-172009415cf3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:51.609587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:51.689127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f952b2-cc67-454b-a31d-27452b7f03ca None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:51.710376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7337cc9-f962-4281-a1e4-172009415cf3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:51.730614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887dc9f-a06d-422f-b248-22415384e3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:51.731295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887dc9f-a06d-422f-b248-22415384e3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.731578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887dc9f-a06d-422f-b248-22415384e3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.731854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887dc9f-a06d-422f-b248-22415384e3c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.732634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887dc9f-a06d-422f-b248-22415384e3c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:51.732942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887dc9f-a06d-422f-b248-22415384e3c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:51.734455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887dc9f-a06d-422f-b248-22415384e3c8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:51.747706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89f1b0-eca6-4a64-9ba7-35edcae5cf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:51.748552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89f1b0-eca6-4a64-9ba7-35edcae5cf05 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.749100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 245/491] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:51:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:51.749292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89f1b0-eca6-4a64-9ba7-35edcae5cf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.749522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89f1b0-eca6-4a64-9ba7-35edcae5cf05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.750014 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887dc9f-a06d-422f-b248-22415384e3c8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:51.760738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:51.762043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89f1b0-eca6-4a64-9ba7-35edcae5cf05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:51.763355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89f1b0-eca6-4a64-9ba7-35edcae5cf05 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-6d89f1b0-eca6-4a64-9ba7-35edcae5cf05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:51.781333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:51.782365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:51.783405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:51.783405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:51.783405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:51.783405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:51.783405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 247/492] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:51.787979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89f1b0-eca6-4a64-9ba7-35edcae5cf05 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:51.848944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:51.872823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887dc9f-a06d-422f-b248-22415384e3c8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:51.899949 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89f1b0-eca6-4a64-9ba7-35edcae5cf05 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:51.928917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102f9bf-b5db-4ffe-b837-0a40766bb7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:51.930538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102f9bf-b5db-4ffe-b837-0a40766bb7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.931096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102f9bf-b5db-4ffe-b837-0a40766bb7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.931516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102f9bf-b5db-4ffe-b837-0a40766bb7f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.932429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102f9bf-b5db-4ffe-b837-0a40766bb7f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:51.932894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102f9bf-b5db-4ffe-b837-0a40766bb7f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:51.934779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102f9bf-b5db-4ffe-b837-0a40766bb7f6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:51.938382 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faecb4f3-bf92-428a-8f34-ddd9d81c47fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:51.939181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecb4f3-bf92-428a-8f34-ddd9d81c47fa None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.939461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecb4f3-bf92-428a-8f34-ddd9d81c47fa None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.939735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faecb4f3-bf92-428a-8f34-ddd9d81c47fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.943437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 246/493] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:51:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:51.944171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102f9bf-b5db-4ffe-b837-0a40766bb7f6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:51.951704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecb4f3-bf92-428a-8f34-ddd9d81c47fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:51.953187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faecb4f3-bf92-428a-8f34-ddd9d81c47fa None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-faecb4f3-bf92-428a-8f34-ddd9d81c47fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:51.971527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:51.972583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:51.973476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:51.973476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:51.973476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:51.973476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:51.973476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 248/494] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:51.974033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18f7f-cd32-4411-b55b-cebbb1d11576 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:51.974321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18f7f-cd32-4411-b55b-cebbb1d11576 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:51.974599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18f7f-cd32-4411-b55b-cebbb1d11576 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:51.980443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 249/495] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:51.980443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18f7f-cd32-4411-b55b-cebbb1d11576 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.023444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:52.063268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102f9bf-b5db-4ffe-b837-0a40766bb7f6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:52.084920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18f7f-cd32-4411-b55b-cebbb1d11576 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:52.096510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:52.100750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe554fb-536b-4caa-9ba0-5168508d56bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:52.101419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe554fb-536b-4caa-9ba0-5168508d56bf None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.101751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe554fb-536b-4caa-9ba0-5168508d56bf None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.102080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe554fb-536b-4caa-9ba0-5168508d56bf None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.102804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe554fb-536b-4caa-9ba0-5168508d56bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:52.103111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe554fb-536b-4caa-9ba0-5168508d56bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:52.104517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe554fb-536b-4caa-9ba0-5168508d56bf None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:52.121720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 247/496] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:52.121720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe554fb-536b-4caa-9ba0-5168508d56bf None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.129509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2b592-710a-4ca8-9bf8-9d8b2fd9eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:52.130241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b592-710a-4ca8-9bf8-9d8b2fd9eba4 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.130524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b592-710a-4ca8-9bf8-9d8b2fd9eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.130798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b592-710a-4ca8-9bf8-9d8b2fd9eba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.142644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2b592-710a-4ca8-9bf8-9d8b2fd9eba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:52.144111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2b592-710a-4ca8-9bf8-9d8b2fd9eba4 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-b8f2b592-710a-4ca8-9bf8-9d8b2fd9eba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:52.165300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:52.166174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:52.172341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:52.172341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:52.172341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:52.172341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:52.172341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 250/497] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:52.172341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9930f-81c9-4553-8322-7466f37fee2b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.172341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9930f-81c9-4553-8322-7466f37fee2b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.172341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9930f-81c9-4553-8322-7466f37fee2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.197328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-88a9930f-81c9-4553-8322-7466f37fee2b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:52.198587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-88a9930f-81c9-4553-8322-7466f37fee2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:52.246199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe554fb-536b-4caa-9ba0-5168508d56bf None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:52.273327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:52.274280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 251/498] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:52.274980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9930f-81c9-4553-8322-7466f37fee2b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.289110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58902f-4a62-49ca-abf5-794a635ffa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:52.290004 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58902f-4a62-49ca-abf5-794a635ffa69 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.290497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58902f-4a62-49ca-abf5-794a635ffa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.290602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58902f-4a62-49ca-abf5-794a635ffa69 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.291282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58902f-4a62-49ca-abf5-794a635ffa69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:52.291638 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58902f-4a62-49ca-abf5-794a635ffa69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:52.293084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58902f-4a62-49ca-abf5-794a635ffa69 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:52.304371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 248/499] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:52.305723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58902f-4a62-49ca-abf5-794a635ffa69 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.366854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:52.376576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9930f-81c9-4553-8322-7466f37fee2b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:52.407130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58902f-4a62-49ca-abf5-794a635ffa69 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:52.409877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75d499-b8c3-4035-9993-7ee39b397409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:52.410667 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d499-b8c3-4035-9993-7ee39b397409 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.411210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d499-b8c3-4035-9993-7ee39b397409 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.411486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d499-b8c3-4035-9993-7ee39b397409 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.419178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75d499-b8c3-4035-9993-7ee39b397409 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:52.420652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75d499-b8c3-4035-9993-7ee39b397409 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-ab75d499-b8c3-4035-9993-7ee39b397409 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:51:52.436244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:52.437219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:52.438105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 252/500] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jun 8 01:51:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:52.438736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f862be-2e67-4507-9405-ad98835fcd80 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.439022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f862be-2e67-4507-9405-ad98835fcd80 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.439296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f862be-2e67-4507-9405-ad98835fcd80 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.440541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 253/501] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:52.444672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f862be-2e67-4507-9405-ad98835fcd80 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.446843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e736b-cbaf-4eca-945d-7974f534b232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:52.447514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e736b-cbaf-4eca-945d-7974f534b232 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.447800 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e736b-cbaf-4eca-945d-7974f534b232 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.448077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e736b-cbaf-4eca-945d-7974f534b232 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.448839 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e736b-cbaf-4eca-945d-7974f534b232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:52.449149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e736b-cbaf-4eca-945d-7974f534b232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:52.450533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e736b-cbaf-4eca-945d-7974f534b232 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:52.462875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 249/502] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:52.463511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e736b-cbaf-4eca-945d-7974f534b232 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.515725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:52.552125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f862be-2e67-4507-9405-ad98835fcd80 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:52.566456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e736b-cbaf-4eca-945d-7974f534b232 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:52.585586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f6f2b-0c9e-457e-9f2a-523aa97e05d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:52.587010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f6f2b-0c9e-457e-9f2a-523aa97e05d7 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.587293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f6f2b-0c9e-457e-9f2a-523aa97e05d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.587574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f6f2b-0c9e-457e-9f2a-523aa97e05d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.597835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f6f2b-0c9e-457e-9f2a-523aa97e05d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:52.600842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c344d8-d463-4588-9e91-779a00a53d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:52.601634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c344d8-d463-4588-9e91-779a00a53d78 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.602152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c344d8-d463-4588-9e91-779a00a53d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.602451 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c344d8-d463-4588-9e91-779a00a53d78 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-2d7f6f2b-0c9e-457e-9f2a-523aa97e05d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:52.605704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:52.606739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:52.607630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 254/503] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:51:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:52.608272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f6f2b-0c9e-457e-9f2a-523aa97e05d7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.609632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c344d8-d463-4588-9e91-779a00a53d78 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:52.611075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c344d8-d463-4588-9e91-779a00a53d78 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:52.625907 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-92c344d8-d463-4588-9e91-779a00a53d78 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:51:52.627144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:52.628238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:52.629402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 250/504] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:52.630071 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92ab66-630b-4d7a-b2b0-4cdf9292d695 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.630147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92ab66-630b-4d7a-b2b0-4cdf9292d695 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.630386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92ab66-630b-4d7a-b2b0-4cdf9292d695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.646798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3d92ab66-630b-4d7a-b2b0-4cdf9292d695 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:52.647965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3d92ab66-630b-4d7a-b2b0-4cdf9292d695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:52.715369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 251/505] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:52.716131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f6f2b-0c9e-457e-9f2a-523aa97e05d7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:52.716242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92ab66-630b-4d7a-b2b0-4cdf9292d695 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.754750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:52.762167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae8797-8e34-4277-8df1-12380999f418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:52.762839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae8797-8e34-4277-8df1-12380999f418 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.763125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae8797-8e34-4277-8df1-12380999f418 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.763388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae8797-8e34-4277-8df1-12380999f418 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.764217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae8797-8e34-4277-8df1-12380999f418 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:52.764559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae8797-8e34-4277-8df1-12380999f418 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:52.766002 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae8797-8e34-4277-8df1-12380999f418 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:52.804275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 255/506] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jun 8 01:51:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:52.805211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae8797-8e34-4277-8df1-12380999f418 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.815638 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92ab66-630b-4d7a-b2b0-4cdf9292d695 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:52.849851 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5986fc-9235-40d2-a377-c994990006a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:52.850637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5986fc-9235-40d2-a377-c994990006a6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.850923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5986fc-9235-40d2-a377-c994990006a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.851200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5986fc-9235-40d2-a377-c994990006a6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.859106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5986fc-9235-40d2-a377-c994990006a6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:52.860588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5986fc-9235-40d2-a377-c994990006a6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:52.874317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5e5986fc-9235-40d2-a377-c994990006a6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:51:52.874928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:52.876247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 252/507] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:51:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:52.877352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5986fc-9235-40d2-a377-c994990006a6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.908638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae8797-8e34-4277-8df1-12380999f418 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:52.945425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1f0f7-5faf-4fa5-bfac-202890fdf718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:52.946334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1f0f7-5faf-4fa5-bfac-202890fdf718 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:52.946618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1f0f7-5faf-4fa5-bfac-202890fdf718 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:52.946894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1f0f7-5faf-4fa5-bfac-202890fdf718 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:52.947632 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1f0f7-5faf-4fa5-bfac-202890fdf718 None admin] RBAC: Adding query filter params (name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:52.947934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1f0f7-5faf-4fa5-bfac-202890fdf718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:52.949356 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1f0f7-5faf-4fa5-bfac-202890fdf718 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:52.957273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 256/508] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:51:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:52.958042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1f0f7-5faf-4fa5-bfac-202890fdf718 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:52.979826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:52.983430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5986fc-9235-40d2-a377-c994990006a6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:53.017700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dd542-7e98-4b53-9967-4c908232927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:53.018364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dd542-7e98-4b53-9967-4c908232927c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:53.018643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dd542-7e98-4b53-9967-4c908232927c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:53.018922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dd542-7e98-4b53-9967-4c908232927c None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:53.019745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dd542-7e98-4b53-9967-4c908232927c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:53.020056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dd542-7e98-4b53-9967-4c908232927c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:53.021513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dd542-7e98-4b53-9967-4c908232927c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:53.058127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1f0f7-5faf-4fa5-bfac-202890fdf718 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:53.059478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 253/509] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jun 8 01:51:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:53.060385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dd542-7e98-4b53-9967-4c908232927c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:53.088010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:53.093288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df952e65-4bd6-4bf7-b1e8-0852fc38b463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:53.094076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df952e65-4bd6-4bf7-b1e8-0852fc38b463 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:53.094360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df952e65-4bd6-4bf7-b1e8-0852fc38b463 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:53.094644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df952e65-4bd6-4bf7-b1e8-0852fc38b463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:53.105971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df952e65-4bd6-4bf7-b1e8-0852fc38b463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:53.107450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df952e65-4bd6-4bf7-b1e8-0852fc38b463 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-df952e65-4bd6-4bf7-b1e8-0852fc38b463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:53.124473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:53.125342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:53.126198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:53.126198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:53.126198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:53.126198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:53.126198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 257/510] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:53.126471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df952e65-4bd6-4bf7-b1e8-0852fc38b463 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:53.164648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dd542-7e98-4b53-9967-4c908232927c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:53.198137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a035aa-97f2-4945-9bae-05f414e78938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:53.198783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a035aa-97f2-4945-9bae-05f414e78938 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:53.199068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a035aa-97f2-4945-9bae-05f414e78938 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:53.199343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a035aa-97f2-4945-9bae-05f414e78938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:53.223136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:53.224595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df952e65-4bd6-4bf7-b1e8-0852fc38b463 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:53.257674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633beed2-a5e2-403f-a183-10ef435698bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:53.258864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633beed2-a5e2-403f-a183-10ef435698bd None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:53.259146 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633beed2-a5e2-403f-a183-10ef435698bd None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:53.259420 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633beed2-a5e2-403f-a183-10ef435698bd None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:53.260245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633beed2-a5e2-403f-a183-10ef435698bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:53.260582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633beed2-a5e2-403f-a183-10ef435698bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:53.262029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633beed2-a5e2-403f-a183-10ef435698bd None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:53.268176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a035aa-97f2-4945-9bae-05f414e78938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:53.269670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a035aa-97f2-4945-9bae-05f414e78938 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:53.279703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 258/511] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jun 8 01:51:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:53.280551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633beed2-a5e2-403f-a183-10ef435698bd None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:53.323008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:53.369351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 254/512] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:53.370191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a035aa-97f2-4945-9bae-05f414e78938 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:53.378955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633beed2-a5e2-403f-a183-10ef435698bd None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:53.412620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2284ac-eaf6-41d7-84a9-dea7f3dfb802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:53.413405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2284ac-eaf6-41d7-84a9-dea7f3dfb802 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:53.413683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2284ac-eaf6-41d7-84a9-dea7f3dfb802 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:53.413955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2284ac-eaf6-41d7-84a9-dea7f3dfb802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:53.424519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2284ac-eaf6-41d7-84a9-dea7f3dfb802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:53.425184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:53.426090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2284ac-eaf6-41d7-84a9-dea7f3dfb802 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-cb2284ac-eaf6-41d7-84a9-dea7f3dfb802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:53.444981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:53.446535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:53.447431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:53.447431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:53.447431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:53.447431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:53.447431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 259/513] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:53.447772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2284ac-eaf6-41d7-84a9-dea7f3dfb802 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:53.470772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a035aa-97f2-4945-9bae-05f414e78938 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:53.505130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cf6ce-77ae-4df2-b495-c9c471f7f32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:53.505810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cf6ce-77ae-4df2-b495-c9c471f7f32e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:53.506093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cf6ce-77ae-4df2-b495-c9c471f7f32e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:53.506367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cf6ce-77ae-4df2-b495-c9c471f7f32e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:53.589849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cf6ce-77ae-4df2-b495-c9c471f7f32e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:53.591252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cf6ce-77ae-4df2-b495-c9c471f7f32e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:53.648882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:53.665052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:53.671949 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2284ac-eaf6-41d7-84a9-dea7f3dfb802 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:53.698422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 255/514] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:53.699265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cf6ce-77ae-4df2-b495-c9c471f7f32e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:53.705855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d547b-040e-4b69-8040-36f578016190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:53.706548 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d547b-040e-4b69-8040-36f578016190 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:53.706831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d547b-040e-4b69-8040-36f578016190 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:53.707111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d547b-040e-4b69-8040-36f578016190 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:53.707835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d547b-040e-4b69-8040-36f578016190 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:53.708142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d547b-040e-4b69-8040-36f578016190 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:53.709568 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d547b-040e-4b69-8040-36f578016190 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:53.721655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 260/515] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:53.722436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d547b-040e-4b69-8040-36f578016190 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:53.798100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cf6ce-77ae-4df2-b495-c9c471f7f32e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:53.823041 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d547b-040e-4b69-8040-36f578016190 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:53.834132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c436f-517c-414d-8f18-8e34129d651b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:53.834790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c436f-517c-414d-8f18-8e34129d651b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:53.835073 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c436f-517c-414d-8f18-8e34129d651b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:53.835347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c436f-517c-414d-8f18-8e34129d651b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:53.865182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb930b-f300-42d0-978d-976b0b75203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:53.865908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb930b-f300-42d0-978d-976b0b75203a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:53.866184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb930b-f300-42d0-978d-976b0b75203a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:53.866460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb930b-f300-42d0-978d-976b0b75203a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:53.870101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:53.877882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb930b-f300-42d0-978d-976b0b75203a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:53.879365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb930b-f300-42d0-978d-976b0b75203a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c2bb930b-f300-42d0-978d-976b0b75203a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:53.902012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:53.902990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:53.903832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:53.903832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:53.903832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:53.903832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:53.903832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 261/516] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:53.904102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb930b-f300-42d0-978d-976b0b75203a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:53.923529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:53.929375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c436f-517c-414d-8f18-8e34129d651b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:53.931513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c436f-517c-414d-8f18-8e34129d651b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.014749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb930b-f300-42d0-978d-976b0b75203a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.031729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 256/517] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:54.032607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c436f-517c-414d-8f18-8e34129d651b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.051068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908293d-18d4-4b13-ba6f-dbacd8453a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.051733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908293d-18d4-4b13-ba6f-dbacd8453a1b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.052020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908293d-18d4-4b13-ba6f-dbacd8453a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.052297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908293d-18d4-4b13-ba6f-dbacd8453a1b None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.053063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908293d-18d4-4b13-ba6f-dbacd8453a1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:54.053379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908293d-18d4-4b13-ba6f-dbacd8453a1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.054760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908293d-18d4-4b13-ba6f-dbacd8453a1b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.066820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 262/518] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:54.067590 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908293d-18d4-4b13-ba6f-dbacd8453a1b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.087843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:54.133101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c436f-517c-414d-8f18-8e34129d651b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.135099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:54.169617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908293d-18d4-4b13-ba6f-dbacd8453a1b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.170581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfefdab-0d54-4e01-987f-bdcb8cdaf533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.171870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfefdab-0d54-4e01-987f-bdcb8cdaf533 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.172206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfefdab-0d54-4e01-987f-bdcb8cdaf533 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.172521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfefdab-0d54-4e01-987f-bdcb8cdaf533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.183473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfefdab-0d54-4e01-987f-bdcb8cdaf533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.184982 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfefdab-0d54-4e01-987f-bdcb8cdaf533 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0dfefdab-0d54-4e01-987f-bdcb8cdaf533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:54.204550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:54.205753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:54.207176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:54.207176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:54.207176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:54.207176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:54.207176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 257/519] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:54.207176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfefdab-0d54-4e01-987f-bdcb8cdaf533 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.215276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c90fb-11c4-48fa-8ab5-f28354868ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.215952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c90fb-11c4-48fa-8ab5-f28354868ca3 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.216233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c90fb-11c4-48fa-8ab5-f28354868ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.217065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c90fb-11c4-48fa-8ab5-f28354868ca3 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.217296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c90fb-11c4-48fa-8ab5-f28354868ca3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.218732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c90fb-11c4-48fa-8ab5-f28354868ca3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.248057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 263/520] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:51:54 2022] POST /identity/v3/users => generated 247 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:51:54.248944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c90fb-11c4-48fa-8ab5-f28354868ca3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.326273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfefdab-0d54-4e01-987f-bdcb8cdaf533 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.338200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:54.364357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c90fb-11c4-48fa-8ab5-f28354868ca3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.371068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c3a4d-fd14-46a0-8818-bb416e368600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.372595 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c3a4d-fd14-46a0-8818-bb416e368600 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.373105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c3a4d-fd14-46a0-8818-bb416e368600 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.373555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c3a4d-fd14-46a0-8818-bb416e368600 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.374840 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c3a4d-fd14-46a0-8818-bb416e368600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:54.375336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c3a4d-fd14-46a0-8818-bb416e368600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.377521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c3a4d-fd14-46a0-8818-bb416e368600 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.389835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:54.395996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 258/521] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:54.395996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c3a4d-fd14-46a0-8818-bb416e368600 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.412300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6ee2a-8c66-4d1b-8148-e4f1913e5092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.413272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6ee2a-8c66-4d1b-8148-e4f1913e5092 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.413551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6ee2a-8c66-4d1b-8148-e4f1913e5092 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.413829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6ee2a-8c66-4d1b-8148-e4f1913e5092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.425881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6ee2a-8c66-4d1b-8148-e4f1913e5092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.427381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6ee2a-8c66-4d1b-8148-e4f1913e5092 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-78f6ee2a-8c66-4d1b-8148-e4f1913e5092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:54.451688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:54.452662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:54.453637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:54.453637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:54.453637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:54.453637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:54.453637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 264/522] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:54.454032 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6ee2a-8c66-4d1b-8148-e4f1913e5092 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.514684 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c3a4d-fd14-46a0-8818-bb416e368600 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.556911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52c49-e96a-4bd6-9d9a-7d3e44347994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.557734 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52c49-e96a-4bd6-9d9a-7d3e44347994 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.558051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52c49-e96a-4bd6-9d9a-7d3e44347994 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.558326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb52c49-e96a-4bd6-9d9a-7d3e44347994 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.565551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb52c49-e96a-4bd6-9d9a-7d3e44347994 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.567004 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb52c49-e96a-4bd6-9d9a-7d3e44347994 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.581991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6ee2a-8c66-4d1b-8148-e4f1913e5092 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2fb52c49-e96a-4bd6-9d9a-7d3e44347994 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:51:54.582148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:54.583010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:54.583904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:54.583904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:54.583904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:51:54.583904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:51:54.583904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:51:54.583904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:51:54.583904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:51:54.583904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:51:54.583904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 08 01:51:54.583904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:54.588216 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 259/523] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:51:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:54.588998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52c49-e96a-4bd6-9d9a-7d3e44347994 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.619413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:54.620370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6f290-7578-4f0d-9aba-d536fce9f7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.621412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6f290-7578-4f0d-9aba-d536fce9f7bc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.621690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6f290-7578-4f0d-9aba-d536fce9f7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.621972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6f290-7578-4f0d-9aba-d536fce9f7bc None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.622692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6f290-7578-4f0d-9aba-d536fce9f7bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:54.623145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6f290-7578-4f0d-9aba-d536fce9f7bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.624390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6f290-7578-4f0d-9aba-d536fce9f7bc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.641870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 265/524] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:54.641870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6f290-7578-4f0d-9aba-d536fce9f7bc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.679412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:54.705965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb52c49-e96a-4bd6-9d9a-7d3e44347994 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.744665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61008f-5762-4d20-bb8b-774f7b129b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.745470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61008f-5762-4d20-bb8b-774f7b129b8d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.745752 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61008f-5762-4d20-bb8b-774f7b129b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.746030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61008f-5762-4d20-bb8b-774f7b129b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.752959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61008f-5762-4d20-bb8b-774f7b129b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.753812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6f290-7578-4f0d-9aba-d536fce9f7bc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.754405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61008f-5762-4d20-bb8b-774f7b129b8d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-ab61008f-5762-4d20-bb8b-774f7b129b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:51:54.769230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:54.770133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:54.770973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:51:54.770973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:51:54.770973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:51:54.770973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:51:54.770973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:51:54.770973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:51:54.770973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:51:54.770973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:54.770973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 260/525] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:54.771448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61008f-5762-4d20-bb8b-774f7b129b8d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.799592 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71b251-b3ce-4f55-a263-cbf352cecd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.800253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71b251-b3ce-4f55-a263-cbf352cecd5d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.800566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71b251-b3ce-4f55-a263-cbf352cecd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.800911 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71b251-b3ce-4f55-a263-cbf352cecd5d None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.801555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71b251-b3ce-4f55-a263-cbf352cecd5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:54.801860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71b251-b3ce-4f55-a263-cbf352cecd5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.803758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71b251-b3ce-4f55-a263-cbf352cecd5d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.819992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 266/526] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:54.821387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71b251-b3ce-4f55-a263-cbf352cecd5d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.867420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:54.880433 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61008f-5762-4d20-bb8b-774f7b129b8d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.924202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372d8e3-64d6-4ebc-a60e-38443fdc528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.925327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372d8e3-64d6-4ebc-a60e-38443fdc528f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.925617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372d8e3-64d6-4ebc-a60e-38443fdc528f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.925894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372d8e3-64d6-4ebc-a60e-38443fdc528f None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.926882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372d8e3-64d6-4ebc-a60e-38443fdc528f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:54.927192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372d8e3-64d6-4ebc-a60e-38443fdc528f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.928648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372d8e3-64d6-4ebc-a60e-38443fdc528f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:54.930537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:54.937696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71b251-b3ce-4f55-a263-cbf352cecd5d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:54.952763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 261/527] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Jun 8 01:51:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:54.953641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372d8e3-64d6-4ebc-a60e-38443fdc528f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:54.974596 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123ba0d-c248-4034-89cd-70aa73d4c052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:54.975396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123ba0d-c248-4034-89cd-70aa73d4c052 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:54.975679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123ba0d-c248-4034-89cd-70aa73d4c052 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:54.976354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123ba0d-c248-4034-89cd-70aa73d4c052 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:54.983456 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123ba0d-c248-4034-89cd-70aa73d4c052 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:54.984933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123ba0d-c248-4034-89cd-70aa73d4c052 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-5123ba0d-c248-4034-89cd-70aa73d4c052 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:51:55.000309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:55.001436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:55.002298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:51:55.002298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:51:55.002298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:51:55.002298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:51:55.002298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:51:55.002298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:51:55.002298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:51:55.002298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:55.002298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 267/528] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:55.002298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123ba0d-c248-4034-89cd-70aa73d4c052 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:55.062574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372d8e3-64d6-4ebc-a60e-38443fdc528f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:55.099838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28946c-d843-4a70-9651-9e7a71d72a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:55.100535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28946c-d843-4a70-9651-9e7a71d72a62 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:55.100821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28946c-d843-4a70-9651-9e7a71d72a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:55.101098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28946c-d843-4a70-9651-9e7a71d72a62 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:55.102000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28946c-d843-4a70-9651-9e7a71d72a62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:55.102315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28946c-d843-4a70-9651-9e7a71d72a62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:55.103209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123ba0d-c248-4034-89cd-70aa73d4c052 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:55.103756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28946c-d843-4a70-9651-9e7a71d72a62 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:55.126660 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 262/529] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:51:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:55.126660 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28946c-d843-4a70-9651-9e7a71d72a62 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:55.135227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:55.144041 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b787d4-dc5f-431d-8259-f92f34b70d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:55.144930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b787d4-dc5f-431d-8259-f92f34b70d98 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:55.145212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b787d4-dc5f-431d-8259-f92f34b70d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:55.145534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b787d4-dc5f-431d-8259-f92f34b70d98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:55.152967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b787d4-dc5f-431d-8259-f92f34b70d98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:55.154460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b787d4-dc5f-431d-8259-f92f34b70d98 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-79b787d4-dc5f-431d-8259-f92f34b70d98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:51:55.168798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:55.169819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:55.170685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:51:55.170685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:51:55.170685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:51:55.170685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:51:55.170685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:51:55.170685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:51:55.170685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:51:55.170685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:55.170685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 268/530] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:55.170685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b787d4-dc5f-431d-8259-f92f34b70d98 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:55.190927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:55.243860 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28946c-d843-4a70-9651-9e7a71d72a62 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:55.279793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af27d44-83eb-45ab-904c-aaf7653e8317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:55.280763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af27d44-83eb-45ab-904c-aaf7653e8317 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:55.281243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af27d44-83eb-45ab-904c-aaf7653e8317 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:55.281755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af27d44-83eb-45ab-904c-aaf7653e8317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:55.281941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b787d4-dc5f-431d-8259-f92f34b70d98 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:55.318175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168eafdc-30c0-420b-8aa7-aa27360521c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:55.318834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168eafdc-30c0-420b-8aa7-aa27360521c7 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:55.319111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168eafdc-30c0-420b-8aa7-aa27360521c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:55.319380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168eafdc-30c0-420b-8aa7-aa27360521c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:55.320853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168eafdc-30c0-420b-8aa7-aa27360521c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:55.320853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168eafdc-30c0-420b-8aa7-aa27360521c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:55.321994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168eafdc-30c0-420b-8aa7-aa27360521c7 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:55.345205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 269/531] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:55.345205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168eafdc-30c0-420b-8aa7-aa27360521c7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:55.363330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af27d44-83eb-45ab-904c-aaf7653e8317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:55.364845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af27d44-83eb-45ab-904c-aaf7653e8317 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:55.373512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:55.423218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:55.450418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168eafdc-30c0-420b-8aa7-aa27360521c7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:55.475255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 263/532] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:55.477141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af27d44-83eb-45ab-904c-aaf7653e8317 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:55.494019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d6df2-d72e-4fdf-b953-3d2d0b091c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:55.494821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d6df2-d72e-4fdf-b953-3d2d0b091c53 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:55.495109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d6df2-d72e-4fdf-b953-3d2d0b091c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:55.495412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d6df2-d72e-4fdf-b953-3d2d0b091c53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:55.496225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d6df2-d72e-4fdf-b953-3d2d0b091c53 None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:55.496573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d6df2-d72e-4fdf-b953-3d2d0b091c53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:55.498016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d6df2-d72e-4fdf-b953-3d2d0b091c53 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:55.504209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 270/533] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:55 2022] GET /identity/v3/role_assignments?user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:55.505044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d6df2-d72e-4fdf-b953-3d2d0b091c53 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:55.620684 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af27d44-83eb-45ab-904c-aaf7653e8317 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:55.630448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:55.644387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d6df2-d72e-4fdf-b953-3d2d0b091c53 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:55.667709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf7670-806c-4004-970f-867dc432dc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:55.668546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7670-806c-4004-970f-867dc432dc3e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:55.668943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7670-806c-4004-970f-867dc432dc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:55.669321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf7670-806c-4004-970f-867dc432dc3e None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:55.670113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf7670-806c-4004-970f-867dc432dc3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:55.670499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf7670-806c-4004-970f-867dc432dc3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:55.672065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf7670-806c-4004-970f-867dc432dc3e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:55.689736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 264/534] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:55.690936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf7670-806c-4004-970f-867dc432dc3e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:55.696104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0506e-6efe-4656-9f55-d78e3219f460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:55.698319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0506e-6efe-4656-9f55-d78e3219f460 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:55.698783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0506e-6efe-4656-9f55-d78e3219f460 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:55.699218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0506e-6efe-4656-9f55-d78e3219f460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:55.700203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0506e-6efe-4656-9f55-d78e3219f460 None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=7d1e01f6f82a4ebbb15d0cb55b963af4) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:55.700687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0506e-6efe-4656-9f55-d78e3219f460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:55.702277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0506e-6efe-4656-9f55-d78e3219f460 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:55.710276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 271/535] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:55 2022] GET /identity/v3/role_assignments?user.id=7d1e01f6f82a4ebbb15d0cb55b963af4&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:55.712049 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0506e-6efe-4656-9f55-d78e3219f460 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:55.720845 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:55.841302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0506e-6efe-4656-9f55-d78e3219f460 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:55.883525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa725330-636e-4fff-b37b-418c388e00c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:55.884654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa725330-636e-4fff-b37b-418c388e00c5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:55.885114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa725330-636e-4fff-b37b-418c388e00c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:55.885561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa725330-636e-4fff-b37b-418c388e00c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:55.886570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa725330-636e-4fff-b37b-418c388e00c5 None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=236226ec865b49febc7713df1c0fab98) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:55.887647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa725330-636e-4fff-b37b-418c388e00c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:55.889413 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa725330-636e-4fff-b37b-418c388e00c5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:55.896910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 272/536] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:55 2022] GET /identity/v3/role_assignments?user.id=236226ec865b49febc7713df1c0fab98&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:55.928875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf7670-806c-4004-970f-867dc432dc3e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:56.014377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a520e5-539e-4507-b364-4f67b3250bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:56.015218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a520e5-539e-4507-b364-4f67b3250bdb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:56.015510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a520e5-539e-4507-b364-4f67b3250bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:56.015792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a520e5-539e-4507-b364-4f67b3250bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:56.037745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a520e5-539e-4507-b364-4f67b3250bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:56.039373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a520e5-539e-4507-b364-4f67b3250bdb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-29a520e5-539e-4507-b364-4f67b3250bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:56.059568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:56.060536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:56.061377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:56.061377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:56.061377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:56.061377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:56.061377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 265/537] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:56.066424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa725330-636e-4fff-b37b-418c388e00c5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:56.176824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:56.199587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa725330-636e-4fff-b37b-418c388e00c5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:56.239899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e13b3f-5893-4217-84d9-d6ef3b1626d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:56.240595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e13b3f-5893-4217-84d9-d6ef3b1626d9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:56.240880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e13b3f-5893-4217-84d9-d6ef3b1626d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:56.241155 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e13b3f-5893-4217-84d9-d6ef3b1626d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:56.241885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e13b3f-5893-4217-84d9-d6ef3b1626d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:56.242329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e13b3f-5893-4217-84d9-d6ef3b1626d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:56.243753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e13b3f-5893-4217-84d9-d6ef3b1626d9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:56.262654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 273/538] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:56.267733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a520e5-539e-4507-b364-4f67b3250bdb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:56.296622 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:56.393520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a520e5-539e-4507-b364-4f67b3250bdb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:56.435822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7535bbb-e867-4d8c-a901-45fffd817dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:56.436656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7535bbb-e867-4d8c-a901-45fffd817dac None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:56.436939 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7535bbb-e867-4d8c-a901-45fffd817dac None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:56.437211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7535bbb-e867-4d8c-a901-45fffd817dac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:56.444342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7535bbb-e867-4d8c-a901-45fffd817dac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:56.446582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7535bbb-e867-4d8c-a901-45fffd817dac None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a7535bbb-e867-4d8c-a901-45fffd817dac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:51:56.468852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:56.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:56.470561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:51:56.470561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:51:56.470561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:51:56.470561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:51:56.470561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:51:56.470561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:51:56.470561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:51:56.470561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:56.470561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 266/539] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:56.475948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e13b3f-5893-4217-84d9-d6ef3b1626d9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:56.595346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e13b3f-5893-4217-84d9-d6ef3b1626d9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:56.621940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:56.638333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d9792-81ce-4c50-a833-53f33bf1907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:56.639006 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d9792-81ce-4c50-a833-53f33bf1907b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:56.639338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d9792-81ce-4c50-a833-53f33bf1907b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:56.639611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d9792-81ce-4c50-a833-53f33bf1907b None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:56.640425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d9792-81ce-4c50-a833-53f33bf1907b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:56.640783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d9792-81ce-4c50-a833-53f33bf1907b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:56.643158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d9792-81ce-4c50-a833-53f33bf1907b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:56.656532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 274/540] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:51:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:56.662331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7535bbb-e867-4d8c-a901-45fffd817dac None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:56.696632 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658d107-bc02-4620-9d12-b93a6e411cdb None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:56.697029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658d107-bc02-4620-9d12-b93a6e411cdb None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:56.697309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658d107-bc02-4620-9d12-b93a6e411cdb None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:56.699586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 275/541] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:51:56.705025 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ba053-1d5d-4d72-a42e-0a08ebf0a6ce None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:56.705318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ba053-1d5d-4d72-a42e-0a08ebf0a6ce None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:56.705600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ba053-1d5d-4d72-a42e-0a08ebf0a6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:56.743166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-fe7ba053-1d5d-4d72-a42e-0a08ebf0a6ce None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:56.745405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-fe7ba053-1d5d-4d72-a42e-0a08ebf0a6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:56.785149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:56.830295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7535bbb-e867-4d8c-a901-45fffd817dac None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:56.878156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 276/542] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:56.889560 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1dbd6-79a8-45a4-bd00-71fe57c7ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:56.890314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1dbd6-79a8-45a4-bd00-71fe57c7ba41 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:56.890597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1dbd6-79a8-45a4-bd00-71fe57c7ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:56.890874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1dbd6-79a8-45a4-bd00-71fe57c7ba41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:56.891770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1dbd6-79a8-45a4-bd00-71fe57c7ba41 None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=d26f4426ba8c43399c4a7ef2f1701f57) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:56.892080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1dbd6-79a8-45a4-bd00-71fe57c7ba41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:56.893612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1dbd6-79a8-45a4-bd00-71fe57c7ba41 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:56.903543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 267/543] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:56 2022] GET /identity/v3/role_assignments?user.id=d26f4426ba8c43399c4a7ef2f1701f57&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:56.907985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ba053-1d5d-4d72-a42e-0a08ebf0a6ce None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:57.074047 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ba053-1d5d-4d72-a42e-0a08ebf0a6ce None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:57.097603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:57.120438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14181dbf-0f16-4a55-a1ec-59ddf309bdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:57.121244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14181dbf-0f16-4a55-a1ec-59ddf309bdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:57.121528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14181dbf-0f16-4a55-a1ec-59ddf309bdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:57.121800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14181dbf-0f16-4a55-a1ec-59ddf309bdd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:57.132465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14181dbf-0f16-4a55-a1ec-59ddf309bdd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-14181dbf-0f16-4a55-a1ec-59ddf309bdd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:57.140748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:57.141619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:57.142452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 277/544] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:51:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:57.148198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1dbd6-79a8-45a4-bd00-71fe57c7ba41 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:57.256075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1dbd6-79a8-45a4-bd00-71fe57c7ba41 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:57.292217 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916899e-82c5-496b-bb5c-cbaf9ba9a7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:57.292926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916899e-82c5-496b-bb5c-cbaf9ba9a7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:57.293205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916899e-82c5-496b-bb5c-cbaf9ba9a7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:57.293474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916899e-82c5-496b-bb5c-cbaf9ba9a7f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:57.294218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916899e-82c5-496b-bb5c-cbaf9ba9a7f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:57.294519 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916899e-82c5-496b-bb5c-cbaf9ba9a7f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:57.295886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916899e-82c5-496b-bb5c-cbaf9ba9a7f3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:57.304503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 268/545] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:51:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:57.313529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14181dbf-0f16-4a55-a1ec-59ddf309bdd4 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:57.430774 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14181dbf-0f16-4a55-a1ec-59ddf309bdd4 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:57.469646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37abb42-2f05-4b4b-abaa-110d64c9fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:57.470446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37abb42-2f05-4b4b-abaa-110d64c9fac7 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:57.470728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37abb42-2f05-4b4b-abaa-110d64c9fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:57.471001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37abb42-2f05-4b4b-abaa-110d64c9fac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:57.482489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37abb42-2f05-4b4b-abaa-110d64c9fac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:57.483957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37abb42-2f05-4b4b-abaa-110d64c9fac7 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-f37abb42-2f05-4b4b-abaa-110d64c9fac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:57.502014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:57.504098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:57.505448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:57.505448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:57.505448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:57.505448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:57.505448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 278/546] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:57.509477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916899e-82c5-496b-bb5c-cbaf9ba9a7f3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:57.513445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:57.615617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916899e-82c5-496b-bb5c-cbaf9ba9a7f3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:57.653954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8c550-07d8-4c0c-a6d8-77c42d4b654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:57.654720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8c550-07d8-4c0c-a6d8-77c42d4b654d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:57.655082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8c550-07d8-4c0c-a6d8-77c42d4b654d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:57.655582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8c550-07d8-4c0c-a6d8-77c42d4b654d None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:57.656459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8c550-07d8-4c0c-a6d8-77c42d4b654d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:57.656818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8c550-07d8-4c0c-a6d8-77c42d4b654d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:57.658794 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8c550-07d8-4c0c-a6d8-77c42d4b654d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:57.672598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 269/547] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:51:57.682038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37abb42-2f05-4b4b-abaa-110d64c9fac7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:57.737465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:57.844499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37abb42-2f05-4b4b-abaa-110d64c9fac7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:57.891769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa06a5-6c19-4b3e-a4c2-5f4a7e988745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:57.892595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa06a5-6c19-4b3e-a4c2-5f4a7e988745 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:57.892880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa06a5-6c19-4b3e-a4c2-5f4a7e988745 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:57.893153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa06a5-6c19-4b3e-a4c2-5f4a7e988745 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:57.906452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa06a5-6c19-4b3e-a4c2-5f4a7e988745 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:57.907886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa06a5-6c19-4b3e-a4c2-5f4a7e988745 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-0eaa06a5-6c19-4b3e-a4c2-5f4a7e988745 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:51:57.932824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:57.934258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:57.935329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 279/548] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:51:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:57.939704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8c550-07d8-4c0c-a6d8-77c42d4b654d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:58.077722 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:58.118650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8c550-07d8-4c0c-a6d8-77c42d4b654d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:58.155046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3444a7-bb46-49fa-8ea9-753a9c69eb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:58.155807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3444a7-bb46-49fa-8ea9-753a9c69eb4f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.156004 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3444a7-bb46-49fa-8ea9-753a9c69eb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.156283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3444a7-bb46-49fa-8ea9-753a9c69eb4f None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.157154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3444a7-bb46-49fa-8ea9-753a9c69eb4f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:58.157465 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3444a7-bb46-49fa-8ea9-753a9c69eb4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:58.158921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3444a7-bb46-49fa-8ea9-753a9c69eb4f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:58.177546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 270/549] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jun 8 01:51:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:58.183788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa06a5-6c19-4b3e-a4c2-5f4a7e988745 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:58.229431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6c255-8e40-49a4-94f6-606ad9c9b748 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.229751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6c255-8e40-49a4-94f6-606ad9c9b748 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.230002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6c255-8e40-49a4-94f6-606ad9c9b748 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.232214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 271/550] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:58.246155 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34e59e-ad1c-4991-bcea-986cc2f47bec None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.246436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34e59e-ad1c-4991-bcea-986cc2f47bec None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.246712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34e59e-ad1c-4991-bcea-986cc2f47bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.272792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2c34e59e-ad1c-4991-bcea-986cc2f47bec None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:58.274607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2c34e59e-ad1c-4991-bcea-986cc2f47bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:58.376801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:58.391780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa06a5-6c19-4b3e-a4c2-5f4a7e988745 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:58.428999 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec31436-16d8-46be-aec9-1f19cbd15a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:58.429787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec31436-16d8-46be-aec9-1f19cbd15a6f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.430056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec31436-16d8-46be-aec9-1f19cbd15a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.430312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec31436-16d8-46be-aec9-1f19cbd15a6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.474323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 272/551] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:51:58.510620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec31436-16d8-46be-aec9-1f19cbd15a6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:58.511990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34e59e-ad1c-4991-bcea-986cc2f47bec None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:58.512102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec31436-16d8-46be-aec9-1f19cbd15a6f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:58.620149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 280/552] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:51:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:51:58.620802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:58.625882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec31436-16d8-46be-aec9-1f19cbd15a6f None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:58.732889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:58.739441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec31436-16d8-46be-aec9-1f19cbd15a6f None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:58.782709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34e59e-ad1c-4991-bcea-986cc2f47bec None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:58.784865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4bdfc-73d7-45df-99c8-35b3f5867262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:58.786997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4bdfc-73d7-45df-99c8-35b3f5867262 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.787460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4bdfc-73d7-45df-99c8-35b3f5867262 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.787895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4bdfc-73d7-45df-99c8-35b3f5867262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.800964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4bdfc-73d7-45df-99c8-35b3f5867262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:58.803596 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4bdfc-73d7-45df-99c8-35b3f5867262 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:58.823376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5c825-700a-4b22-b7f4-9a5e5f0ea4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:58.824302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5c825-700a-4b22-b7f4-9a5e5f0ea4ec None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.824703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5c825-700a-4b22-b7f4-9a5e5f0ea4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.825235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5c825-700a-4b22-b7f4-9a5e5f0ea4ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-18b4bdfc-73d7-45df-99c8-35b3f5867262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:58.827513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:58.828515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:58.829801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:58.829801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:58.829801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:58.829801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:58.829801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 281/553] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:58.830940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b729aa-7c3a-40a9-bcd0-21825e83db48 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.831164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b729aa-7c3a-40a9-bcd0-21825e83db48 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.831709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b729aa-7c3a-40a9-bcd0-21825e83db48 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.833027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 282/554] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:58.834000 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1854-7daa-483a-85c1-91214578ae29 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.834202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1854-7daa-483a-85c1-91214578ae29 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.834480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1854-7daa-483a-85c1-91214578ae29 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.836446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 283/555] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:58.838517 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1854-7daa-483a-85c1-91214578ae29 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:58.840792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5c825-700a-4b22-b7f4-9a5e5f0ea4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d7c5c825-700a-4b22-b7f4-9a5e5f0ea4ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:58.848781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:58.849796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:58.850664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:58.853226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 273/556] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:51:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:58.854854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b62ef-02bc-4787-b2c2-dd1dbbd46c80 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.855593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b62ef-02bc-4787-b2c2-dd1dbbd46c80 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.855883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b62ef-02bc-4787-b2c2-dd1dbbd46c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.873527 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fb8b62ef-02bc-4787-b2c2-dd1dbbd46c80 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:58.874723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fb8b62ef-02bc-4787-b2c2-dd1dbbd46c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:58.884521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:58.949241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1854-7daa-483a-85c1-91214578ae29 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:58.958746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 274/557] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:58.969751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfed12-a7ac-4b77-ac46-334638179f1d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.976879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfed12-a7ac-4b77-ac46-334638179f1d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.976879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfed12-a7ac-4b77-ac46-334638179f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.991624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f3dd2-dd90-48cc-bd54-0f04be8030b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:58.991800 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-36bfed12-a7ac-4b77-ac46-334638179f1d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:51:58.992820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f3dd2-dd90-48cc-bd54-0f04be8030b6 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:58.993107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f3dd2-dd90-48cc-bd54-0f04be8030b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:58.993277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-36bfed12-a7ac-4b77-ac46-334638179f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:51:58.993395 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f3dd2-dd90-48cc-bd54-0f04be8030b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:58.994102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f3dd2-dd90-48cc-bd54-0f04be8030b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:58.994409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f3dd2-dd90-48cc-bd54-0f04be8030b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:58.995780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f3dd2-dd90-48cc-bd54-0f04be8030b6 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:59.012468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 284/558] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:59.012468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f3dd2-dd90-48cc-bd54-0f04be8030b6 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.037598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:59.073786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 275/559] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:51:59.074628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfed12-a7ac-4b77-ac46-334638179f1d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.124101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:59.128456 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f3dd2-dd90-48cc-bd54-0f04be8030b6 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:59.166277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e59f1e-3b9b-443c-8d75-b6d2d1a181f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:59.166947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e59f1e-3b9b-443c-8d75-b6d2d1a181f8 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.167414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e59f1e-3b9b-443c-8d75-b6d2d1a181f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.167543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e59f1e-3b9b-443c-8d75-b6d2d1a181f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.168247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e59f1e-3b9b-443c-8d75-b6d2d1a181f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:59.168599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e59f1e-3b9b-443c-8d75-b6d2d1a181f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:59.169988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e59f1e-3b9b-443c-8d75-b6d2d1a181f8 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:59.182719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 285/560] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:59.182719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e59f1e-3b9b-443c-8d75-b6d2d1a181f8 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.191327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfed12-a7ac-4b77-ac46-334638179f1d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:59.228453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6fd08-bb32-462a-bac5-0eb9b8ab2841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:59.229259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6fd08-bb32-462a-bac5-0eb9b8ab2841 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.229548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6fd08-bb32-462a-bac5-0eb9b8ab2841 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.229826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6fd08-bb32-462a-bac5-0eb9b8ab2841 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.240442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6fd08-bb32-462a-bac5-0eb9b8ab2841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-73d6fd08-bb32-462a-bac5-0eb9b8ab2841 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:59.248449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:59.249407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:59.250258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 276/561] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:59.251423 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6fd08-bb32-462a-bac5-0eb9b8ab2841 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.286661 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:59.299791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e59f1e-3b9b-443c-8d75-b6d2d1a181f8 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:59.339210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0cdb-6174-4dcc-a67b-f0026d9a410b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:59.340016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0cdb-6174-4dcc-a67b-f0026d9a410b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.340293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0cdb-6174-4dcc-a67b-f0026d9a410b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.340607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0cdb-6174-4dcc-a67b-f0026d9a410b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.347765 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf0cdb-6174-4dcc-a67b-f0026d9a410b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:59.349289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf0cdb-6174-4dcc-a67b-f0026d9a410b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-bedf0cdb-6174-4dcc-a67b-f0026d9a410b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:51:59.364480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:59.365414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:59.366330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:51:59.366330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:51:59.366330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:51:59.366330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:51:59.366330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:51:59.366330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:51:59.366330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:51:59.366330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:59.367025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6fd08-bb32-462a-bac5-0eb9b8ab2841 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:59.370358 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 286/562] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:59.371186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0cdb-6174-4dcc-a67b-f0026d9a410b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.396226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:59.405540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bde68-7c6e-4a24-87ce-72cac6cfd5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:59.406348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bde68-7c6e-4a24-87ce-72cac6cfd5bd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.406636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bde68-7c6e-4a24-87ce-72cac6cfd5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.406934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bde68-7c6e-4a24-87ce-72cac6cfd5bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.419952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bde68-7c6e-4a24-87ce-72cac6cfd5bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-864bde68-7c6e-4a24-87ce-72cac6cfd5bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:51:59.427632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:59.428675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 277/563] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:59.429712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bde68-7c6e-4a24-87ce-72cac6cfd5bd None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.477756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0cdb-6174-4dcc-a67b-f0026d9a410b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:59.512666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c6da8-3c45-4acf-a6dd-e08a7cad95ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:59.513392 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c6da8-3c45-4acf-a6dd-e08a7cad95ff None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.513676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c6da8-3c45-4acf-a6dd-e08a7cad95ff None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.513953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c6da8-3c45-4acf-a6dd-e08a7cad95ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.525003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c6da8-3c45-4acf-a6dd-e08a7cad95ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:59.526478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c6da8-3c45-4acf-a6dd-e08a7cad95ff None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:59.537360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-244c6da8-3c45-4acf-a6dd-e08a7cad95ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:59.544992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:59.545900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:59.546781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:59.546781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:59.546781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:59.546781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:51:59.546781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 287/564] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:59.547066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c6da8-3c45-4acf-a6dd-e08a7cad95ff None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.554280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bde68-7c6e-4a24-87ce-72cac6cfd5bd None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:59.590041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458dae0-97d2-4d04-9239-849dc8dbf3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:59.590719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458dae0-97d2-4d04-9239-849dc8dbf3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.591000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458dae0-97d2-4d04-9239-849dc8dbf3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.591280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458dae0-97d2-4d04-9239-849dc8dbf3e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.592020 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458dae0-97d2-4d04-9239-849dc8dbf3e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:59.592326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458dae0-97d2-4d04-9239-849dc8dbf3e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:59.593726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458dae0-97d2-4d04-9239-849dc8dbf3e7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:59.607377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 278/565] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:59.607377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458dae0-97d2-4d04-9239-849dc8dbf3e7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.633862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:59.662262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c6da8-3c45-4acf-a6dd-e08a7cad95ff None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:59.696800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827eba7-e131-4d31-85f5-100dbbbcb84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:59.697461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827eba7-e131-4d31-85f5-100dbbbcb84c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.697748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827eba7-e131-4d31-85f5-100dbbbcb84c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.698027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827eba7-e131-4d31-85f5-100dbbbcb84c None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.698815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827eba7-e131-4d31-85f5-100dbbbcb84c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:59.699123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827eba7-e131-4d31-85f5-100dbbbcb84c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:59.700590 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827eba7-e131-4d31-85f5-100dbbbcb84c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:59.712852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 288/566] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:59.713640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827eba7-e131-4d31-85f5-100dbbbcb84c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.715319 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458dae0-97d2-4d04-9239-849dc8dbf3e7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:59.750331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc9b55-d52e-451d-b297-60a8299cc595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:59.751108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc9b55-d52e-451d-b297-60a8299cc595 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.751498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc9b55-d52e-451d-b297-60a8299cc595 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.751848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc9b55-d52e-451d-b297-60a8299cc595 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.752703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc9b55-d52e-451d-b297-60a8299cc595 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:59.753017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc9b55-d52e-451d-b297-60a8299cc595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:59.754401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc9b55-d52e-451d-b297-60a8299cc595 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:59.762329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 279/567] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:59.763165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc9b55-d52e-451d-b297-60a8299cc595 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.781236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:59.819232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827eba7-e131-4d31-85f5-100dbbbcb84c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:59.853470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11eb77-22e9-490b-a74d-d3ca12b2653a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:59.854125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11eb77-22e9-490b-a74d-d3ca12b2653a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.854406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11eb77-22e9-490b-a74d-d3ca12b2653a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.854681 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11eb77-22e9-490b-a74d-d3ca12b2653a None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.855391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11eb77-22e9-490b-a74d-d3ca12b2653a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:51:59.855700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11eb77-22e9-490b-a74d-d3ca12b2653a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:59.857092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11eb77-22e9-490b-a74d-d3ca12b2653a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:59.858500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:51:59.862837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc9b55-d52e-451d-b297-60a8299cc595 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:51:59.869627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 289/568] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:51:59.870889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53df46-6bdb-467d-8d83-8000a5b1b193 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.871177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53df46-6bdb-467d-8d83-8000a5b1b193 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.871455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53df46-6bdb-467d-8d83-8000a5b1b193 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.876522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 290/569] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:51:59.876522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53df46-6bdb-467d-8d83-8000a5b1b193 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.900165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f96b38-4556-4b74-81d9-8b8b3c04e314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:51:59.900994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f96b38-4556-4b74-81d9-8b8b3c04e314 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:51:59.901281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f96b38-4556-4b74-81d9-8b8b3c04e314 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:51:59.901553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f96b38-4556-4b74-81d9-8b8b3c04e314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:51:59.913186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f96b38-4556-4b74-81d9-8b8b3c04e314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:51:59.915920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f96b38-4556-4b74-81d9-8b8b3c04e314 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-06f96b38-4556-4b74-81d9-8b8b3c04e314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:51:59.933691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:51:59.934671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:51:59.935540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:51:59.935540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:51:59.935540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:51:59.935540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:51:59.939552 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 280/570] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:51:59.940302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f96b38-4556-4b74-81d9-8b8b3c04e314 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:51:59.975551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53df46-6bdb-467d-8d83-8000a5b1b193 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.006777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:00.011017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de1ac0-3294-44ac-b27d-4cc0b2af78e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.011665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de1ac0-3294-44ac-b27d-4cc0b2af78e9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.011944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de1ac0-3294-44ac-b27d-4cc0b2af78e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.012217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de1ac0-3294-44ac-b27d-4cc0b2af78e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.013065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de1ac0-3294-44ac-b27d-4cc0b2af78e9 None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=fc299258fe814636880cf595fc2904b4) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:00.013420 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de1ac0-3294-44ac-b27d-4cc0b2af78e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:00.014795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de1ac0-3294-44ac-b27d-4cc0b2af78e9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:00.020309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 291/571] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/role_assignments?user.id=fc299258fe814636880cf595fc2904b4&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:00.021097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de1ac0-3294-44ac-b27d-4cc0b2af78e9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:00.044376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f96b38-4556-4b74-81d9-8b8b3c04e314 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.081218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510aef8-c59d-4c5a-b9b3-4d81019aa92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.082197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510aef8-c59d-4c5a-b9b3-4d81019aa92d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.082629 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510aef8-c59d-4c5a-b9b3-4d81019aa92d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.083095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510aef8-c59d-4c5a-b9b3-4d81019aa92d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.089619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:00.098393 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510aef8-c59d-4c5a-b9b3-4d81019aa92d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:00.099894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510aef8-c59d-4c5a-b9b3-4d81019aa92d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b510aef8-c59d-4c5a-b9b3-4d81019aa92d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:00.118630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:00.119602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:00.120529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:00.120529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:00.120529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:00.120529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:00.120529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 281/572] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:00.121338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442c7e7-0c1a-482d-9f1e-a885729ae3b7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.121585 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442c7e7-0c1a-482d-9f1e-a885729ae3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.121877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442c7e7-0c1a-482d-9f1e-a885729ae3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.140358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d442c7e7-0c1a-482d-9f1e-a885729ae3b7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:00.141616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d442c7e7-0c1a-482d-9f1e-a885729ae3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:00.142986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de1ac0-3294-44ac-b27d-4cc0b2af78e9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.182118 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffae565-82d8-4931-80f5-d496bd6076c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.183042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae565-82d8-4931-80f5-d496bd6076c5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.183394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae565-82d8-4931-80f5-d496bd6076c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.183735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae565-82d8-4931-80f5-d496bd6076c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.192891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffae565-82d8-4931-80f5-d496bd6076c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:00.194662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffae565-82d8-4931-80f5-d496bd6076c5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-0ffae565-82d8-4931-80f5-d496bd6076c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:00.213148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:00.214168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:00.215103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:00.215103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:00.215103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:00.215103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:00.215103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:00.215103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:00.215103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:00.215103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:00.215103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:00.215103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:00.218149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 292/573] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:00.218149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffae565-82d8-4931-80f5-d496bd6076c5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:00.220233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 282/574] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:00.220694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442c7e7-0c1a-482d-9f1e-a885729ae3b7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:00.279652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:00.330217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffae565-82d8-4931-80f5-d496bd6076c5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.340588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442c7e7-0c1a-482d-9f1e-a885729ae3b7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.349333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:00.367891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a8ebd-3606-4ed2-aeb0-b7c01f0a2367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.368657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a8ebd-3606-4ed2-aeb0-b7c01f0a2367 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.368939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a8ebd-3606-4ed2-aeb0-b7c01f0a2367 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.369212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a8ebd-3606-4ed2-aeb0-b7c01f0a2367 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.369933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a8ebd-3606-4ed2-aeb0-b7c01f0a2367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:00.370238 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a8ebd-3606-4ed2-aeb0-b7c01f0a2367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:00.371610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a8ebd-3606-4ed2-aeb0-b7c01f0a2367 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:00.380766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0d140-3732-4790-88ff-d63e95baac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.382083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0d140-3732-4790-88ff-d63e95baac62 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.382577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0d140-3732-4790-88ff-d63e95baac62 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.383090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0d140-3732-4790-88ff-d63e95baac62 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.383647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 293/575] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:00.384224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0d140-3732-4790-88ff-d63e95baac62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:00.384527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a8ebd-3606-4ed2-aeb0-b7c01f0a2367 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:00.384913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0d140-3732-4790-88ff-d63e95baac62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:00.386356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0d140-3732-4790-88ff-d63e95baac62 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:00.401830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 283/576] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:00.403132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0d140-3732-4790-88ff-d63e95baac62 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:00.492778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a8ebd-3606-4ed2-aeb0-b7c01f0a2367 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.522183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0d140-3732-4790-88ff-d63e95baac62 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.529098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddac174-0e88-4590-8df5-cf3222a3d149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.531761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddac174-0e88-4590-8df5-cf3222a3d149 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.531761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddac174-0e88-4590-8df5-cf3222a3d149 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.531761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddac174-0e88-4590-8df5-cf3222a3d149 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.531761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddac174-0e88-4590-8df5-cf3222a3d149 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:00.532338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddac174-0e88-4590-8df5-cf3222a3d149 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:00.533668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:00.534160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddac174-0e88-4590-8df5-cf3222a3d149 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:00.553181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 294/577] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:00.554000 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddac174-0e88-4590-8df5-cf3222a3d149 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:00.565090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cab6f-864c-4d96-88b6-7b2103b68460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.565895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cab6f-864c-4d96-88b6-7b2103b68460 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.566177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cab6f-864c-4d96-88b6-7b2103b68460 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.566449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cab6f-864c-4d96-88b6-7b2103b68460 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.577654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659cab6f-864c-4d96-88b6-7b2103b68460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-659cab6f-864c-4d96-88b6-7b2103b68460 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:00.585610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:00.587298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:00.588277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:00.588952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 284/578] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:00.589736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cab6f-864c-4d96-88b6-7b2103b68460 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:00.597489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:00.668046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddac174-0e88-4590-8df5-cf3222a3d149 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.708718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b6d4e-8568-4747-a8fe-6612f81fa01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.708929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cab6f-864c-4d96-88b6-7b2103b68460 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.709525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b6d4e-8568-4747-a8fe-6612f81fa01d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.709808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b6d4e-8568-4747-a8fe-6612f81fa01d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.710083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b6d4e-8568-4747-a8fe-6612f81fa01d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.722183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b6d4e-8568-4747-a8fe-6612f81fa01d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:00.723883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b6d4e-8568-4747-a8fe-6612f81fa01d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-9a3b6d4e-8568-4747-a8fe-6612f81fa01d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:00.739094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:00.740253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 295/579] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:00.741232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b6d4e-8568-4747-a8fe-6612f81fa01d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:00.752082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cebb03-847d-42fa-a467-8cadf140da4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.752915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cebb03-847d-42fa-a467-8cadf140da4e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.753197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cebb03-847d-42fa-a467-8cadf140da4e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.753467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cebb03-847d-42fa-a467-8cadf140da4e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.761778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cebb03-847d-42fa-a467-8cadf140da4e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:00.763228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cebb03-847d-42fa-a467-8cadf140da4e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-c8cebb03-847d-42fa-a467-8cadf140da4e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:00.778205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:00.779206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:00.780195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 285/580] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:00.780858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cebb03-847d-42fa-a467-8cadf140da4e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:00.817690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:00.849134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:00.857242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b6d4e-8568-4747-a8fe-6612f81fa01d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.895175 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cebb03-847d-42fa-a467-8cadf140da4e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:00.897258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd887bf9-f9b7-4f4a-97b8-21e5341049f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.899709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd887bf9-f9b7-4f4a-97b8-21e5341049f8 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.899992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd887bf9-f9b7-4f4a-97b8-21e5341049f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.900267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd887bf9-f9b7-4f4a-97b8-21e5341049f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.917353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd887bf9-f9b7-4f4a-97b8-21e5341049f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:00.918924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd887bf9-f9b7-4f4a-97b8-21e5341049f8 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:00.935077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed41a40-2e70-4257-9427-a7d28b809fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:00.935751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed41a40-2e70-4257-9427-a7d28b809fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:00.936038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed41a40-2e70-4257-9427-a7d28b809fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:00.936318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed41a40-2e70-4257-9427-a7d28b809fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-dd887bf9-f9b7-4f4a-97b8-21e5341049f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:00.936923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:00.937768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:00.943490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:00.943490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:00.943490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:00.943490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:00.943725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed41a40-2e70-4257-9427-a7d28b809fa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:00.943725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed41a40-2e70-4257-9427-a7d28b809fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:00.943865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 296/581] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:00.943934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed41a40-2e70-4257-9427-a7d28b809fa7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:00.944021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd887bf9-f9b7-4f4a-97b8-21e5341049f8 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:00.951492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 286/582] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:00.952210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed41a40-2e70-4257-9427-a7d28b809fa7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.052272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:01.062135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd887bf9-f9b7-4f4a-97b8-21e5341049f8 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.064220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed41a40-2e70-4257-9427-a7d28b809fa7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.098999 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f893d-23c3-401f-8241-e9c0b34bfc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.099600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f893d-23c3-401f-8241-e9c0b34bfc5f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.099887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f893d-23c3-401f-8241-e9c0b34bfc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.100164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f893d-23c3-401f-8241-e9c0b34bfc5f None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.101038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f893d-23c3-401f-8241-e9c0b34bfc5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:01.101930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f893d-23c3-401f-8241-e9c0b34bfc5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.103522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f893d-23c3-401f-8241-e9c0b34bfc5f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:01.109894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e5a02-59a6-4202-9e5c-cd66bcfb0a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.110574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e5a02-59a6-4202-9e5c-cd66bcfb0a46 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.110855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e5a02-59a6-4202-9e5c-cd66bcfb0a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.111128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e5a02-59a6-4202-9e5c-cd66bcfb0a46 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.111951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e5a02-59a6-4202-9e5c-cd66bcfb0a46 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:01.112256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e5a02-59a6-4202-9e5c-cd66bcfb0a46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.113715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e5a02-59a6-4202-9e5c-cd66bcfb0a46 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:01.126683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 297/583] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:01.126683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f893d-23c3-401f-8241-e9c0b34bfc5f None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.137335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 287/584] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jun 8 01:52:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:01.137821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:01.138179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e5a02-59a6-4202-9e5c-cd66bcfb0a46 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.243236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f893d-23c3-401f-8241-e9c0b34bfc5f None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.246796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e5a02-59a6-4202-9e5c-cd66bcfb0a46 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.280996 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bf781-ddc4-4221-a34f-1997aaf59e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.281659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bf781-ddc4-4221-a34f-1997aaf59e94 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.281944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bf781-ddc4-4221-a34f-1997aaf59e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.282222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bf781-ddc4-4221-a34f-1997aaf59e94 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.282537 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc977-4647-4918-9a58-5481ed3ddb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.282943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bf781-ddc4-4221-a34f-1997aaf59e94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:01.283252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bf781-ddc4-4221-a34f-1997aaf59e94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.283360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc977-4647-4918-9a58-5481ed3ddb91 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.283613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc977-4647-4918-9a58-5481ed3ddb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.283893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc977-4647-4918-9a58-5481ed3ddb91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.284655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bf781-ddc4-4221-a34f-1997aaf59e94 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:01.294496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cc977-4647-4918-9a58-5481ed3ddb91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.295951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cc977-4647-4918-9a58-5481ed3ddb91 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:01.301469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 298/585] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:01.301469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bf781-ddc4-4221-a34f-1997aaf59e94 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-3a4cc977-4647-4918-9a58-5481ed3ddb91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:01.319582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:01.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:01.326947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:01.326947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:01.326947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:01.326947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:01.326947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 288/586] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:01.326947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc977-4647-4918-9a58-5481ed3ddb91 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.326947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:01.375789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:01.417959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bf781-ddc4-4221-a34f-1997aaf59e94 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.430485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cc977-4647-4918-9a58-5481ed3ddb91 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.452993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b1292-b28c-4830-bb06-56a30b019d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.453901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b1292-b28c-4830-bb06-56a30b019d44 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.454186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b1292-b28c-4830-bb06-56a30b019d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.454581 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b1292-b28c-4830-bb06-56a30b019d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.465595 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01466ce8-157c-43f8-b563-591c4ea28458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.466391 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01466ce8-157c-43f8-b563-591c4ea28458 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.466682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01466ce8-157c-43f8-b563-591c4ea28458 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.466783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b1292-b28c-4830-bb06-56a30b019d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.466961 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01466ce8-157c-43f8-b563-591c4ea28458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.468436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b1292-b28c-4830-bb06-56a30b019d44 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:01.477461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01466ce8-157c-43f8-b563-591c4ea28458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.478907 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01466ce8-157c-43f8-b563-591c4ea28458 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-292b1292-b28c-4830-bb06-56a30b019d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:01.486287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:01.487174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:01.488073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:01.488073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:01.488073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:01.488073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:01.488073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 299/587] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:01.488354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b1292-b28c-4830-bb06-56a30b019d44 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-01466ce8-157c-43f8-b563-591c4ea28458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:01.501046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:01.501932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:01.502835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:01.502835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:01.502835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:01.502835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:01.506714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 289/588] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:01.507357 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01466ce8-157c-43f8-b563-591c4ea28458 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.568199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:01.597077 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b1292-b28c-4830-bb06-56a30b019d44 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.615331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:01.628858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01466ce8-157c-43f8-b563-591c4ea28458 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.635750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9f74-3055-4de2-ac0b-c2e9a292b512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.637032 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9f74-3055-4de2-ac0b-c2e9a292b512 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.637321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9f74-3055-4de2-ac0b-c2e9a292b512 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.637598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9f74-3055-4de2-ac0b-c2e9a292b512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.644967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a9f74-3055-4de2-ac0b-c2e9a292b512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.646421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a9f74-3055-4de2-ac0b-c2e9a292b512 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-783a9f74-3055-4de2-ac0b-c2e9a292b512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:01.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:01.662795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:01.664332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:01.664332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:01.664332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:01.664332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:01.664332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:01.664332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:01.664332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:01.664332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:01.664332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 300/589] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:01.664332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9f74-3055-4de2-ac0b-c2e9a292b512 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.673533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e173e7e-5c4a-44c8-a77d-df374297bd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.674219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e173e7e-5c4a-44c8-a77d-df374297bd09 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.674502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e173e7e-5c4a-44c8-a77d-df374297bd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.674777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e173e7e-5c4a-44c8-a77d-df374297bd09 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.675505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e173e7e-5c4a-44c8-a77d-df374297bd09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:01.675812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e173e7e-5c4a-44c8-a77d-df374297bd09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.677856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e173e7e-5c4a-44c8-a77d-df374297bd09 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:01.691398 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 290/590] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:01.693717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e173e7e-5c4a-44c8-a77d-df374297bd09 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.777186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9f74-3055-4de2-ac0b-c2e9a292b512 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.798818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e173e7e-5c4a-44c8-a77d-df374297bd09 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.813599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:01.816693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae19af0-cf25-4d4e-818d-9a89676134c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.817777 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae19af0-cf25-4d4e-818d-9a89676134c0 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.817906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae19af0-cf25-4d4e-818d-9a89676134c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.818167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae19af0-cf25-4d4e-818d-9a89676134c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.818897 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae19af0-cf25-4d4e-818d-9a89676134c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:01.819205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae19af0-cf25-4d4e-818d-9a89676134c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.820648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae19af0-cf25-4d4e-818d-9a89676134c0 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:01.836831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169058c-964c-4b29-8ad3-7da1eacbc33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.837438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 301/591] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:01.837885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169058c-964c-4b29-8ad3-7da1eacbc33f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.838158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169058c-964c-4b29-8ad3-7da1eacbc33f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.838259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae19af0-cf25-4d4e-818d-9a89676134c0 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.838437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169058c-964c-4b29-8ad3-7da1eacbc33f None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.839154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169058c-964c-4b29-8ad3-7da1eacbc33f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:01.839464 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169058c-964c-4b29-8ad3-7da1eacbc33f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.840992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169058c-964c-4b29-8ad3-7da1eacbc33f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:01.858636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 291/592] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:01.858636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169058c-964c-4b29-8ad3-7da1eacbc33f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:01.884975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:01.953037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae19af0-cf25-4d4e-818d-9a89676134c0 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.969394 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169058c-964c-4b29-8ad3-7da1eacbc33f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:01.990963 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634e603-71ea-46a8-9f35-cc8352a6f4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:01.991667 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634e603-71ea-46a8-9f35-cc8352a6f4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:01.992649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634e603-71ea-46a8-9f35-cc8352a6f4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:01.993115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634e603-71ea-46a8-9f35-cc8352a6f4e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:01.994052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634e603-71ea-46a8-9f35-cc8352a6f4e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:01.994370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634e603-71ea-46a8-9f35-cc8352a6f4e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:01.996204 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634e603-71ea-46a8-9f35-cc8352a6f4e0 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:02.007376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdaced-2168-40be-977c-d1399553ddaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:02.008339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdaced-2168-40be-977c-d1399553ddaf None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.008655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdaced-2168-40be-977c-d1399553ddaf None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.008938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdaced-2168-40be-977c-d1399553ddaf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.012509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 302/593] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:02.013572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634e603-71ea-46a8-9f35-cc8352a6f4e0 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:02.018697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdaced-2168-40be-977c-d1399553ddaf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:02.020165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdaced-2168-40be-977c-d1399553ddaf None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2abdaced-2168-40be-977c-d1399553ddaf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:02.035459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:02.036745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 292/594] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:02.037700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdaced-2168-40be-977c-d1399553ddaf None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:02.074214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:02.121681 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:02.127137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634e603-71ea-46a8-9f35-cc8352a6f4e0 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:02.145009 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdaced-2168-40be-977c-d1399553ddaf None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:02.166452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ae25-64a5-45a0-ad0f-68d47dfd515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:02.167531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ae25-64a5-45a0-ad0f-68d47dfd515b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.167989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ae25-64a5-45a0-ad0f-68d47dfd515b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.168314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ae25-64a5-45a0-ad0f-68d47dfd515b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.178733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42ae25-64a5-45a0-ad0f-68d47dfd515b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:02.180193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42ae25-64a5-45a0-ad0f-68d47dfd515b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:02.185856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccbb7f-5a5b-4882-a62e-283f93902803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:02.186663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccbb7f-5a5b-4882-a62e-283f93902803 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.186946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccbb7f-5a5b-4882-a62e-283f93902803 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.187219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccbb7f-5a5b-4882-a62e-283f93902803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.194829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccbb7f-5a5b-4882-a62e-283f93902803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:02.197126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daccbb7f-5a5b-4882-a62e-283f93902803 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-2f42ae25-64a5-45a0-ad0f-68d47dfd515b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:02.200892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:02.201753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:02.202635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:02.202635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:02.202635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:02.202635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:02.202635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:02.202635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:02.202635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:02.202635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:02.202635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 303/595] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:02.203715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ae25-64a5-45a0-ad0f-68d47dfd515b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-daccbb7f-5a5b-4882-a62e-283f93902803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:02.217307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:02.218241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:02.219160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:02.219160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:02.219160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:02.219160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:02.219160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:02.219160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:02.219160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:02.219160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:02.219160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 293/596] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:02.219659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccbb7f-5a5b-4882-a62e-283f93902803 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:02.318400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:02.319429 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ae25-64a5-45a0-ad0f-68d47dfd515b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:02.327349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccbb7f-5a5b-4882-a62e-283f93902803 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:02.355663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebca1c-3964-44b5-ac81-3312c86e34c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:02.356791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebca1c-3964-44b5-ac81-3312c86e34c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.357076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebca1c-3964-44b5-ac81-3312c86e34c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.357352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebca1c-3964-44b5-ac81-3312c86e34c3 None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/0ef930dc7b7a4fb29f7b32e464e9eb3b/roles/32b18412b6c944b8899ef24b902885d2` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.361120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01b44-95f8-4e58-9f2b-8628c6bdbe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:02.361783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01b44-95f8-4e58-9f2b-8628c6bdbe9d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.362069 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01b44-95f8-4e58-9f2b-8628c6bdbe9d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.362347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01b44-95f8-4e58-9f2b-8628c6bdbe9d None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.363179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f01b44-95f8-4e58-9f2b-8628c6bdbe9d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:02.363488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f01b44-95f8-4e58-9f2b-8628c6bdbe9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:02.364960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f01b44-95f8-4e58-9f2b-8628c6bdbe9d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:02.372040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebca1c-3964-44b5-ac81-3312c86e34c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=0ef930dc7b7a4fb29f7b32e464e9eb3b, role_id=32b18412b6c944b8899ef24b902885d2)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:02.373532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebca1c-3964-44b5-ac81-3312c86e34c3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:02.385619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 294/597] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Jun 8 01:52:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:02.386488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01b44-95f8-4e58-9f2b-8628c6bdbe9d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:02.400510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 304/598] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:52:02 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/0ef930dc7b7a4fb29f7b32e464e9eb3b/roles/32b18412b6c944b8899ef24b902885d2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:52:02.400510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebca1c-3964-44b5-ac81-3312c86e34c3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:02.402363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:02.513585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebca1c-3964-44b5-ac81-3312c86e34c3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:02.517021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01b44-95f8-4e58-9f2b-8628c6bdbe9d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:02.551637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c45e0d-2231-42bd-a9fe-e5df882bfcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:02.552322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c45e0d-2231-42bd-a9fe-e5df882bfcae None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.552638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c45e0d-2231-42bd-a9fe-e5df882bfcae None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.552919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c45e0d-2231-42bd-a9fe-e5df882bfcae None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.553727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c45e0d-2231-42bd-a9fe-e5df882bfcae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:02.554048 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c45e0d-2231-42bd-a9fe-e5df882bfcae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:02.555851 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982688ca-c9c9-404d-9a4b-a1a65348d42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:02.556110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c45e0d-2231-42bd-a9fe-e5df882bfcae None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:02.556565 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982688ca-c9c9-404d-9a4b-a1a65348d42b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.556857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982688ca-c9c9-404d-9a4b-a1a65348d42b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.557132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982688ca-c9c9-404d-9a4b-a1a65348d42b None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.558064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982688ca-c9c9-404d-9a4b-a1a65348d42b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:02.558377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982688ca-c9c9-404d-9a4b-a1a65348d42b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:02.559974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982688ca-c9c9-404d-9a4b-a1a65348d42b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:02.574849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 305/599] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:02.574849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd45fc-52aa-4c01-b5fb-acdcaab02aa9 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.574849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd45fc-52aa-4c01-b5fb-acdcaab02aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.574849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd45fc-52aa-4c01-b5fb-acdcaab02aa9 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.574849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 306/600] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:02.574849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd45fc-52aa-4c01-b5fb-acdcaab02aa9 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:02.580263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 295/601] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:02.584538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982688ca-c9c9-404d-9a4b-a1a65348d42b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:02.604898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:02.654827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:02.696019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd45fc-52aa-4c01-b5fb-acdcaab02aa9 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:02.704170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982688ca-c9c9-404d-9a4b-a1a65348d42b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:02.739285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf46075-2bb0-4306-b5ee-92dbcd7b1bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:02.740200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf46075-2bb0-4306-b5ee-92dbcd7b1bf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.740512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf46075-2bb0-4306-b5ee-92dbcd7b1bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.740789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf46075-2bb0-4306-b5ee-92dbcd7b1bf8 None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/236226ec865b49febc7713df1c0fab98/roles/32b18412b6c944b8899ef24b902885d2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.751877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ddb96-17f1-41f6-b35b-bd492f8b7854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:02.752713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ddb96-17f1-41f6-b35b-bd492f8b7854 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.752997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ddb96-17f1-41f6-b35b-bd492f8b7854 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.753273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ddb96-17f1-41f6-b35b-bd492f8b7854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.759011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf46075-2bb0-4306-b5ee-92dbcd7b1bf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=236226ec865b49febc7713df1c0fab98, role_id=32b18412b6c944b8899ef24b902885d2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:02.760495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf46075-2bb0-4306-b5ee-92dbcd7b1bf8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:02.765416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ddb96-17f1-41f6-b35b-bd492f8b7854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:02.766880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ddb96-17f1-41f6-b35b-bd492f8b7854 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:02.784435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 296/602] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:52:02 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/236226ec865b49febc7713df1c0fab98/roles/32b18412b6c944b8899ef24b902885d2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:52:02.784435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae869a17-4bdb-4844-a65c-ce0eeb3b4807 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:02.784435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae869a17-4bdb-4844-a65c-ce0eeb3b4807 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:02.784435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae869a17-4bdb-4844-a65c-ce0eeb3b4807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c14ddb96-17f1-41f6-b35b-bd492f8b7854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:02.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:02.797788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:02.798884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:02.798884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:02.798884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:02.798884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:02.799867 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 307/603] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:02.800840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ddb96-17f1-41f6-b35b-bd492f8b7854 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:02.808717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ae869a17-4bdb-4844-a65c-ce0eeb3b4807 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:02.810285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ae869a17-4bdb-4844-a65c-ce0eeb3b4807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:02.886683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:02.902302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 297/604] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:02.903645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae869a17-4bdb-4844-a65c-ce0eeb3b4807 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:02.948190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ddb96-17f1-41f6-b35b-bd492f8b7854 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:02.977910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:02.999412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c56607-5368-4b65-b919-b517424a0bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:03.000616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c56607-5368-4b65-b919-b517424a0bfc None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:03.001177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c56607-5368-4b65-b919-b517424a0bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:03.001600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c56607-5368-4b65-b919-b517424a0bfc None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/7d1e01f6f82a4ebbb15d0cb55b963af4/roles/32b18412b6c944b8899ef24b902885d2` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:03.022468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae869a17-4bdb-4844-a65c-ce0eeb3b4807 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:03.025015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c56607-5368-4b65-b919-b517424a0bfc None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=7d1e01f6f82a4ebbb15d0cb55b963af4, role_id=32b18412b6c944b8899ef24b902885d2)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:03.026648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c56607-5368-4b65-b919-b517424a0bfc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:03.044585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 308/605] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:52:02 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/7d1e01f6f82a4ebbb15d0cb55b963af4/roles/32b18412b6c944b8899ef24b902885d2 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:52:03.045409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c56607-5368-4b65-b919-b517424a0bfc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:03.065074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094efed1-fc06-4de3-bb47-b005d16d45c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:03.066079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094efed1-fc06-4de3-bb47-b005d16d45c8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:03.066544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094efed1-fc06-4de3-bb47-b005d16d45c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:03.066997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094efed1-fc06-4de3-bb47-b005d16d45c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:03.068028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094efed1-fc06-4de3-bb47-b005d16d45c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:03.068571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094efed1-fc06-4de3-bb47-b005d16d45c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:03.070791 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094efed1-fc06-4de3-bb47-b005d16d45c8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:03.088516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 298/606] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:03.095417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094efed1-fc06-4de3-bb47-b005d16d45c8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:03.175789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c56607-5368-4b65-b919-b517424a0bfc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:03.181877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:03.220067 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc635f7-d3f0-44f3-8e5d-6bcb959e4915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:03.221072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc635f7-d3f0-44f3-8e5d-6bcb959e4915 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:03.221434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc635f7-d3f0-44f3-8e5d-6bcb959e4915 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:03.221851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc635f7-d3f0-44f3-8e5d-6bcb959e4915 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:03.234146 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc635f7-d3f0-44f3-8e5d-6bcb959e4915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-3fc635f7-d3f0-44f3-8e5d-6bcb959e4915 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:03.243388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:03.244533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:03.245379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 309/607] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:03.254589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094efed1-fc06-4de3-bb47-b005d16d45c8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:03.255285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc635f7-d3f0-44f3-8e5d-6bcb959e4915 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:03.311718 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f36f69-721a-4a28-8eec-fdbd3b53616b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:03.312875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f36f69-721a-4a28-8eec-fdbd3b53616b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:03.313341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f36f69-721a-4a28-8eec-fdbd3b53616b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:03.313792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f36f69-721a-4a28-8eec-fdbd3b53616b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:03.322781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:03.325231 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f36f69-721a-4a28-8eec-fdbd3b53616b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:03.327531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f36f69-721a-4a28-8eec-fdbd3b53616b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-57f36f69-721a-4a28-8eec-fdbd3b53616b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:03.347992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:03.348987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:03.349874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:03.349874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:03.349874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:03.349874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:03.349874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:03.349874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:03.349874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:03.349874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:03.353074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 299/608] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:03.358692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f36f69-721a-4a28-8eec-fdbd3b53616b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:03.406043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc635f7-d3f0-44f3-8e5d-6bcb959e4915 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:03.442594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d9c5f-e144-4e21-989f-dfaf820cde4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:03.443345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d9c5f-e144-4e21-989f-dfaf820cde4f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:03.443721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d9c5f-e144-4e21-989f-dfaf820cde4f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:03.445044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d9c5f-e144-4e21-989f-dfaf820cde4f None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:03.446547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d9c5f-e144-4e21-989f-dfaf820cde4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:03.446547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d9c5f-e144-4e21-989f-dfaf820cde4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:03.448179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d9c5f-e144-4e21-989f-dfaf820cde4f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:03.457925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 310/609] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:52:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:03.463876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d9c5f-e144-4e21-989f-dfaf820cde4f None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:03.473549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f36f69-721a-4a28-8eec-fdbd3b53616b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:03.487741 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:03.522503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10747c15-2819-4891-856d-2f81d00bf4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:03.524291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10747c15-2819-4891-856d-2f81d00bf4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:03.524829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10747c15-2819-4891-856d-2f81d00bf4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:03.525340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10747c15-2819-4891-856d-2f81d00bf4d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:03.526332 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10747c15-2819-4891-856d-2f81d00bf4d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:03.526828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10747c15-2819-4891-856d-2f81d00bf4d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:03.528593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10747c15-2819-4891-856d-2f81d00bf4d0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:03.556858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 300/610] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:03.562859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10747c15-2819-4891-856d-2f81d00bf4d0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:03.580444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d9c5f-e144-4e21-989f-dfaf820cde4f None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:03.600852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:03.647101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd92c19-3d5b-4b7d-9025-b69a4dcad8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:03.648263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd92c19-3d5b-4b7d-9025-b69a4dcad8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:03.648618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd92c19-3d5b-4b7d-9025-b69a4dcad8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:03.648958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd92c19-3d5b-4b7d-9025-b69a4dcad8c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:03.676426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd92c19-3d5b-4b7d-9025-b69a4dcad8c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:03.678181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd92c19-3d5b-4b7d-9025-b69a4dcad8c6 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:03.686194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10747c15-2819-4891-856d-2f81d00bf4d0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-cbd92c19-3d5b-4b7d-9025-b69a4dcad8c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:03.707034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:03.707952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:03.708987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:03.708987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:03.708987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:03.708987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:03.715654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 311/611] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:03.721915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd92c19-3d5b-4b7d-9025-b69a4dcad8c6 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:03.735556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef010165-5781-4434-929a-ff7d2a83c2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:03.736764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef010165-5781-4434-929a-ff7d2a83c2ef None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:03.737214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef010165-5781-4434-929a-ff7d2a83c2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:03.737636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef010165-5781-4434-929a-ff7d2a83c2ef None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/d26f4426ba8c43399c4a7ef2f1701f57/roles/32b18412b6c944b8899ef24b902885d2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:03.754962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef010165-5781-4434-929a-ff7d2a83c2ef None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=d26f4426ba8c43399c4a7ef2f1701f57, role_id=32b18412b6c944b8899ef24b902885d2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:03.756933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef010165-5781-4434-929a-ff7d2a83c2ef None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:03.782781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 301/612] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:52:03 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/d26f4426ba8c43399c4a7ef2f1701f57/roles/32b18412b6c944b8899ef24b902885d2 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 01:52:03.965005 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd92c19-3d5b-4b7d-9025-b69a4dcad8c6 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:04.024150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d33c5-379d-4c59-90cf-dff5184d2573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:04.025015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d33c5-379d-4c59-90cf-dff5184d2573 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:04.025298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d33c5-379d-4c59-90cf-dff5184d2573 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:04.025574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d33c5-379d-4c59-90cf-dff5184d2573 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:04.026300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d33c5-379d-4c59-90cf-dff5184d2573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:04.026606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d33c5-379d-4c59-90cf-dff5184d2573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:04.027979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d33c5-379d-4c59-90cf-dff5184d2573 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:04.051099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 312/613] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:04.056993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef010165-5781-4434-929a-ff7d2a83c2ef None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:04.090255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:04.176887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef010165-5781-4434-929a-ff7d2a83c2ef None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:04.225441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f35af-ddb7-4d23-b0e6-fe2250f254bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:04.226342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f35af-ddb7-4d23-b0e6-fe2250f254bb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:04.226911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f35af-ddb7-4d23-b0e6-fe2250f254bb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:04.227202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f35af-ddb7-4d23-b0e6-fe2250f254bb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:04.239349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f35af-ddb7-4d23-b0e6-fe2250f254bb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:04.240954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f35af-ddb7-4d23-b0e6-fe2250f254bb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-c88f35af-ddb7-4d23-b0e6-fe2250f254bb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:04.262027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:04.263039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:04.263973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 302/614] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:04.269734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d33c5-379d-4c59-90cf-dff5184d2573 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:04.279757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:04.385108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d33c5-379d-4c59-90cf-dff5184d2573 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:04.425323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70ff12-1d88-4306-b208-0874c5359297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:04.426136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70ff12-1d88-4306-b208-0874c5359297 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:04.426309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70ff12-1d88-4306-b208-0874c5359297 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:04.426521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70ff12-1d88-4306-b208-0874c5359297 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:04.427366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70ff12-1d88-4306-b208-0874c5359297 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:04.427679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70ff12-1d88-4306-b208-0874c5359297 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:04.429151 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70ff12-1d88-4306-b208-0874c5359297 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:04.456725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 313/615] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jun 8 01:52:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:04.460917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f35af-ddb7-4d23-b0e6-fe2250f254bb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:04.602989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:04.634065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f35af-ddb7-4d23-b0e6-fe2250f254bb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:04.678518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448c976-26e4-40a4-acb1-60e7b8f355d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:04.679737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448c976-26e4-40a4-acb1-60e7b8f355d1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:04.680233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448c976-26e4-40a4-acb1-60e7b8f355d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:04.680730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7448c976-26e4-40a4-acb1-60e7b8f355d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:04.697869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448c976-26e4-40a4-acb1-60e7b8f355d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:04.699583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7448c976-26e4-40a4-acb1-60e7b8f355d1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-7448c976-26e4-40a4-acb1-60e7b8f355d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:04.722260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:04.723194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:04.724109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:04.724109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:04.724109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:04.724109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:04.726580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 303/616] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:04.731654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70ff12-1d88-4306-b208-0874c5359297 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:04.825598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:04.873835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70ff12-1d88-4306-b208-0874c5359297 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:04.913205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d854455-c5b2-49c8-89bc-b898d2ca4bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:04.914077 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d854455-c5b2-49c8-89bc-b898d2ca4bda None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:04.914380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d854455-c5b2-49c8-89bc-b898d2ca4bda None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:04.914660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d854455-c5b2-49c8-89bc-b898d2ca4bda None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:04.915393 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d854455-c5b2-49c8-89bc-b898d2ca4bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:04.915701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d854455-c5b2-49c8-89bc-b898d2ca4bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:04.917122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d854455-c5b2-49c8-89bc-b898d2ca4bda None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:04.935122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 314/617] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:04.939592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448c976-26e4-40a4-acb1-60e7b8f355d1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:05.019802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925488f6-76c8-468b-97d6-ce8bab52404c None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:05.024499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925488f6-76c8-468b-97d6-ce8bab52404c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:05.024499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925488f6-76c8-468b-97d6-ce8bab52404c None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:05.024499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 315/618] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:05.027669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2044c-f325-4b15-baf5-902409da631e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:05.027956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2044c-f325-4b15-baf5-902409da631e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:05.028237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2044c-f325-4b15-baf5-902409da631e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:05.045476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b9f2044c-f325-4b15-baf5-902409da631e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:05.046668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b9f2044c-f325-4b15-baf5-902409da631e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:05.121207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7448c976-26e4-40a4-acb1-60e7b8f355d1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:05.172601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 316/619] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:05.187935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c34d0c-e8a0-4d3b-b56d-182bcdb86230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:05.188816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c34d0c-e8a0-4d3b-b56d-182bcdb86230 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:05.189098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c34d0c-e8a0-4d3b-b56d-182bcdb86230 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:05.189368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c34d0c-e8a0-4d3b-b56d-182bcdb86230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:05.199631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c34d0c-e8a0-4d3b-b56d-182bcdb86230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:05.201096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c34d0c-e8a0-4d3b-b56d-182bcdb86230 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:05.204756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-c5c34d0c-e8a0-4d3b-b56d-182bcdb86230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:05.225502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:05.226753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:05.227691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:05.227691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:05.227691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:05.227691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:05.227691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:05.227691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:05.227691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:05.227691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:05.228146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 304/620] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:05.229779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2044c-f325-4b15-baf5-902409da631e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:05.241940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c34d0c-e8a0-4d3b-b56d-182bcdb86230 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:05.326938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:05.389173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2044c-f325-4b15-baf5-902409da631e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:05.449821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c34d0c-e8a0-4d3b-b56d-182bcdb86230 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:05.456911 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ad53a-61e3-45ef-9b34-2002fc7dce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:05.457723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ad53a-61e3-45ef-9b34-2002fc7dce82 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:05.463842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ad53a-61e3-45ef-9b34-2002fc7dce82 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:05.464211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ad53a-61e3-45ef-9b34-2002fc7dce82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:05.476661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ad53a-61e3-45ef-9b34-2002fc7dce82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-6c3ad53a-61e3-45ef-9b34-2002fc7dce82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:05.485613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:05.486571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:05.487703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 317/621] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:05.495853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ad53a-61e3-45ef-9b34-2002fc7dce82 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:05.518736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf2600-20eb-4065-90b1-becdb44fbf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:05.520754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf2600-20eb-4065-90b1-becdb44fbf5a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:05.521067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf2600-20eb-4065-90b1-becdb44fbf5a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:05.521356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf2600-20eb-4065-90b1-becdb44fbf5a None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:05.523762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf2600-20eb-4065-90b1-becdb44fbf5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:05.524172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf2600-20eb-4065-90b1-becdb44fbf5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:05.526614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf2600-20eb-4065-90b1-becdb44fbf5a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:05.553930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 305/622] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:05.560860 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf2600-20eb-4065-90b1-becdb44fbf5a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:05.633880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:05.665770 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ad53a-61e3-45ef-9b34-2002fc7dce82 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:05.684507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:05.691983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf2600-20eb-4065-90b1-becdb44fbf5a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:05.717912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101d6a6-baa6-46e7-a189-5da696b148d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:05.718576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101d6a6-baa6-46e7-a189-5da696b148d2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:05.718869 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101d6a6-baa6-46e7-a189-5da696b148d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:05.719145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101d6a6-baa6-46e7-a189-5da696b148d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:05.720211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101d6a6-baa6-46e7-a189-5da696b148d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:05.720702 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101d6a6-baa6-46e7-a189-5da696b148d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:05.722565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101d6a6-baa6-46e7-a189-5da696b148d2 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:05.734195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813bc0d-71c6-4fe8-883e-20aaa953f738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:05.734384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 318/623] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:52:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:05.735516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd208f45-9995-4875-8209-0cd69d3fe401 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:05.735805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd208f45-9995-4875-8209-0cd69d3fe401 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:05.736036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813bc0d-71c6-4fe8-883e-20aaa953f738 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:05.736138 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd208f45-9995-4875-8209-0cd69d3fe401 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:05.736724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813bc0d-71c6-4fe8-883e-20aaa953f738 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:05.737223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813bc0d-71c6-4fe8-883e-20aaa953f738 None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/fc299258fe814636880cf595fc2904b4/roles/32b18412b6c944b8899ef24b902885d2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:05.741487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 319/624] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:05.744356 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839211-4510-4158-86ba-f2de4cb9c0f2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:05.744680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839211-4510-4158-86ba-f2de4cb9c0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:05.744959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839211-4510-4158-86ba-f2de4cb9c0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:05.760789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813bc0d-71c6-4fe8-883e-20aaa953f738 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=fc299258fe814636880cf595fc2904b4, role_id=32b18412b6c944b8899ef24b902885d2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:05.762548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813bc0d-71c6-4fe8-883e-20aaa953f738 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:05.769378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-17839211-4510-4158-86ba-f2de4cb9c0f2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:05.770582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-17839211-4510-4158-86ba-f2de4cb9c0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:05.784611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 306/625] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:52:05 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/fc299258fe814636880cf595fc2904b4/roles/32b18412b6c944b8899ef24b902885d2 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:52:05.786348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813bc0d-71c6-4fe8-883e-20aaa953f738 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:05.846497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 320/626] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:05.897908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17839211-4510-4158-86ba-f2de4cb9c0f2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:05.934080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813bc0d-71c6-4fe8-883e-20aaa953f738 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:05.937149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:05.978517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9addcca-b9f3-42d8-a640-547de038e7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:05.979805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9addcca-b9f3-42d8-a640-547de038e7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:05.980296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9addcca-b9f3-42d8-a640-547de038e7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:05.980801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9addcca-b9f3-42d8-a640-547de038e7c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:05.993051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9addcca-b9f3-42d8-a640-547de038e7c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:05.994943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9addcca-b9f3-42d8-a640-547de038e7c6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d9addcca-b9f3-42d8-a640-547de038e7c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:06.016116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:06.017113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:06.018016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:06.018016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:06.018016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:06.018016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:06.020376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 307/627] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:06.025796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9addcca-b9f3-42d8-a640-547de038e7c6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:06.110286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17839211-4510-4158-86ba-f2de4cb9c0f2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:06.118529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:06.147472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9addcca-b9f3-42d8-a640-547de038e7c6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:06.153223 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79556843-9c84-4173-8ae8-764f0b2b3697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:06.153610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556843-9c84-4173-8ae8-764f0b2b3697 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.153895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556843-9c84-4173-8ae8-764f0b2b3697 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.154178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556843-9c84-4173-8ae8-764f0b2b3697 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.166727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79556843-9c84-4173-8ae8-764f0b2b3697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-79556843-9c84-4173-8ae8-764f0b2b3697 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:06.174655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:06.175620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:06.176527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 321/628] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:06.181390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea1a4-719d-4dee-9bf1-2920d763cde6 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.181682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea1a4-719d-4dee-9bf1-2920d763cde6 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.181970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea1a4-719d-4dee-9bf1-2920d763cde6 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.183572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 322/629] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:06.188769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534aa5d-7b0b-4db2-a819-19051d7fee7f None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.189069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534aa5d-7b0b-4db2-a819-19051d7fee7f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.189512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534aa5d-7b0b-4db2-a819-19051d7fee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.194668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1dc9c8-a4e1-470b-9918-8519aebae80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:06.196134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dc9c8-a4e1-470b-9918-8519aebae80a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.196662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dc9c8-a4e1-470b-9918-8519aebae80a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.197142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1dc9c8-a4e1-470b-9918-8519aebae80a None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.198060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1dc9c8-a4e1-470b-9918-8519aebae80a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:06.198591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1dc9c8-a4e1-470b-9918-8519aebae80a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:06.200181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1dc9c8-a4e1-470b-9918-8519aebae80a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:06.207514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-4534aa5d-7b0b-4db2-a819-19051d7fee7f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:06.208737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4534aa5d-7b0b-4db2-a819-19051d7fee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:06.215102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 308/630] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:06.216196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1dc9c8-a4e1-470b-9918-8519aebae80a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:06.236132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:06.281173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 323/631] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:52:06.282068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534aa5d-7b0b-4db2-a819-19051d7fee7f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:06.332632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1dc9c8-a4e1-470b-9918-8519aebae80a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:06.371021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb30010-0bc3-4160-834c-17022022369d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:06.372086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb30010-0bc3-4160-834c-17022022369d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.372622 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb30010-0bc3-4160-834c-17022022369d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.373106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb30010-0bc3-4160-834c-17022022369d None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.374076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb30010-0bc3-4160-834c-17022022369d None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:06.374589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb30010-0bc3-4160-834c-17022022369d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:06.376179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb30010-0bc3-4160-834c-17022022369d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:06.385824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 309/632] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:52:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:06.386934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb30010-0bc3-4160-834c-17022022369d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:06.411396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:06.418763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534aa5d-7b0b-4db2-a819-19051d7fee7f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:06.458546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c94f4-d5df-4bc3-b17e-632f74bb60cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:06.459348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c94f4-d5df-4bc3-b17e-632f74bb60cd None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.459590 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c94f4-d5df-4bc3-b17e-632f74bb60cd None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.459846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c94f4-d5df-4bc3-b17e-632f74bb60cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.468321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c94f4-d5df-4bc3-b17e-632f74bb60cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:06.469822 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c94f4-d5df-4bc3-b17e-632f74bb60cd None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:06.485971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-ba7c94f4-d5df-4bc3-b17e-632f74bb60cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:06.487351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:06.488294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:06.489288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 324/633] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jun 8 01:52:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:06.489894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c94f4-d5df-4bc3-b17e-632f74bb60cd None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:06.514850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb30010-0bc3-4160-834c-17022022369d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:06.552413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6381d-2e9b-4d67-a454-2103ff7cdfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:06.553966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6381d-2e9b-4d67-a454-2103ff7cdfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.554450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6381d-2e9b-4d67-a454-2103ff7cdfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.554949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6381d-2e9b-4d67-a454-2103ff7cdfa2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.566185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6381d-2e9b-4d67-a454-2103ff7cdfa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-18d6381d-2e9b-4d67-a454-2103ff7cdfa2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:06.574306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:06.575257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:06.576108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:06.578354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 310/634] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:06.579286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6381d-2e9b-4d67-a454-2103ff7cdfa2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:06.600813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c94f4-d5df-4bc3-b17e-632f74bb60cd None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:06.638405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcf7dc-a76c-44cf-a905-0e5b13a872bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:06.639209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcf7dc-a76c-44cf-a905-0e5b13a872bc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.639495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcf7dc-a76c-44cf-a905-0e5b13a872bc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.639774 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcf7dc-a76c-44cf-a905-0e5b13a872bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.650464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcf7dc-a76c-44cf-a905-0e5b13a872bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:06.651922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcf7dc-a76c-44cf-a905-0e5b13a872bc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-8fbcf7dc-a76c-44cf-a905-0e5b13a872bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:06.670455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:06.671421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:06.672322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:06.672322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:06.672322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:06.672322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:06.672322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 325/635] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:06.673186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcf7dc-a76c-44cf-a905-0e5b13a872bc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:06.682205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:06.696703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6381d-2e9b-4d67-a454-2103ff7cdfa2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:06.731671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04113a-d303-4f6a-97e4-21217b13499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:06.732353 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04113a-d303-4f6a-97e4-21217b13499e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.732669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04113a-d303-4f6a-97e4-21217b13499e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.732947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04113a-d303-4f6a-97e4-21217b13499e None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.733787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04113a-d303-4f6a-97e4-21217b13499e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:06.734158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04113a-d303-4f6a-97e4-21217b13499e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:06.735552 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04113a-d303-4f6a-97e4-21217b13499e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:06.744735 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:06.759969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 311/636] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jun 8 01:52:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:06.759969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc30b5-8130-43c2-9f9c-58a8123dd26c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.759969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc30b5-8130-43c2-9f9c-58a8123dd26c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.759969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc30b5-8130-43c2-9f9c-58a8123dd26c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.759969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 312/637] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:06.759969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc30b5-8130-43c2-9f9c-58a8123dd26c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:06.805312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcf7dc-a76c-44cf-a905-0e5b13a872bc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:06.844836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdc559-b075-42ed-9ae2-e247f5972e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:06.845932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdc559-b075-42ed-9ae2-e247f5972e9c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.845932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdc559-b075-42ed-9ae2-e247f5972e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.846158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdc559-b075-42ed-9ae2-e247f5972e9c None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.846911 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdc559-b075-42ed-9ae2-e247f5972e9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:06.847222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdc559-b075-42ed-9ae2-e247f5972e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:06.848625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdc559-b075-42ed-9ae2-e247f5972e9c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:06.861861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 326/638] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:52:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:06.861861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdc559-b075-42ed-9ae2-e247f5972e9c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:06.880706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc30b5-8130-43c2-9f9c-58a8123dd26c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:06.926831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc26ab-9777-420b-99c2-1f3d5e9f1cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:06.927574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc26ab-9777-420b-99c2-1f3d5e9f1cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.927932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc26ab-9777-420b-99c2-1f3d5e9f1cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.928210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc26ab-9777-420b-99c2-1f3d5e9f1cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.928962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc26ab-9777-420b-99c2-1f3d5e9f1cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:06.929492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc26ab-9777-420b-99c2-1f3d5e9f1cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:06.931170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc26ab-9777-420b-99c2-1f3d5e9f1cd0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:06.933966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:06.943249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 313/639] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:06.944304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5149f9-1bfc-4a27-87d7-a37c464cb098 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:06.944621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5149f9-1bfc-4a27-87d7-a37c464cb098 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:06.945409 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5149f9-1bfc-4a27-87d7-a37c464cb098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:06.963139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4d5149f9-1bfc-4a27-87d7-a37c464cb098 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:06.964314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4d5149f9-1bfc-4a27-87d7-a37c464cb098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:06.977091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdc559-b075-42ed-9ae2-e247f5972e9c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:07.013574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57897e27-18c0-46f4-8a8f-a39fdef4a8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:07.014227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57897e27-18c0-46f4-8a8f-a39fdef4a8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:07.014521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57897e27-18c0-46f4-8a8f-a39fdef4a8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:07.014802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57897e27-18c0-46f4-8a8f-a39fdef4a8f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:07.021185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:07.037348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 314/640] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:07.041262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5149f9-1bfc-4a27-87d7-a37c464cb098 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:07.110455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57897e27-18c0-46f4-8a8f-a39fdef4a8f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:07.112050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57897e27-18c0-46f4-8a8f-a39fdef4a8f9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:07.146978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5149f9-1bfc-4a27-87d7-a37c464cb098 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:07.180842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:07.188720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313998b-8274-4ba7-b56c-5fa2eda4111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:07.189535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313998b-8274-4ba7-b56c-5fa2eda4111f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:07.189824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313998b-8274-4ba7-b56c-5fa2eda4111f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:07.190101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313998b-8274-4ba7-b56c-5fa2eda4111f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:07.204781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313998b-8274-4ba7-b56c-5fa2eda4111f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:07.206242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313998b-8274-4ba7-b56c-5fa2eda4111f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:07.222906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 327/641] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:52:07.222906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57897e27-18c0-46f4-8a8f-a39fdef4a8f9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-6313998b-8274-4ba7-b56c-5fa2eda4111f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:07.224141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:07.225056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:07.225996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:07.225996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:07.225996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:07.225996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:07.225996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 315/642] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:07.227647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313998b-8274-4ba7-b56c-5fa2eda4111f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:07.277071 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:07.326824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57897e27-18c0-46f4-8a8f-a39fdef4a8f9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:07.344563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313998b-8274-4ba7-b56c-5fa2eda4111f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:07.361817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af61bfe-21ee-4733-b13d-32c6ff024487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:07.362628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af61bfe-21ee-4733-b13d-32c6ff024487 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:07.362916 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af61bfe-21ee-4733-b13d-32c6ff024487 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:07.363191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af61bfe-21ee-4733-b13d-32c6ff024487 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:07.371898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af61bfe-21ee-4733-b13d-32c6ff024487 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:07.373391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af61bfe-21ee-4733-b13d-32c6ff024487 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-6af61bfe-21ee-4733-b13d-32c6ff024487 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:07.390446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:07.391855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 328/643] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:07.392865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af61bfe-21ee-4733-b13d-32c6ff024487 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:07.409308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb94f29-7486-47ff-8516-752972cb7085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:07.410426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb94f29-7486-47ff-8516-752972cb7085 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:07.411191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb94f29-7486-47ff-8516-752972cb7085 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:07.411535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb94f29-7486-47ff-8516-752972cb7085 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:07.422832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb94f29-7486-47ff-8516-752972cb7085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:07.426771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-fdb94f29-7486-47ff-8516-752972cb7085 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:07.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:07.431612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:07.432570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 316/644] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:07.433961 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb94f29-7486-47ff-8516-752972cb7085 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:07.504870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af61bfe-21ee-4733-b13d-32c6ff024487 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:07.537826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:07.541668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4415f5-0950-4e8a-859f-80a69047e424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:07.542473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4415f5-0950-4e8a-859f-80a69047e424 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:07.542760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4415f5-0950-4e8a-859f-80a69047e424 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:07.543038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4415f5-0950-4e8a-859f-80a69047e424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:07.544235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb94f29-7486-47ff-8516-752972cb7085 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:07.553620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4415f5-0950-4e8a-859f-80a69047e424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:07.555802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4415f5-0950-4e8a-859f-80a69047e424 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-6a4415f5-0950-4e8a-859f-80a69047e424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:07.573574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:07.574488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:07.575385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:07.575385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:07.575385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:07.575385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:07.575385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 329/645] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:07.575673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4415f5-0950-4e8a-859f-80a69047e424 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:07.582754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0412c13-b29b-414d-af28-6279ea20db29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:07.583427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0412c13-b29b-414d-af28-6279ea20db29 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:07.583711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0412c13-b29b-414d-af28-6279ea20db29 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:07.583985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0412c13-b29b-414d-af28-6279ea20db29 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:07.584769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0412c13-b29b-414d-af28-6279ea20db29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:07.585081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0412c13-b29b-414d-af28-6279ea20db29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:07.586522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0412c13-b29b-414d-af28-6279ea20db29 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:07.603531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 317/646] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:07.603531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0412c13-b29b-414d-af28-6279ea20db29 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:07.678708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:07.708932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0412c13-b29b-414d-af28-6279ea20db29 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:07.712369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4415f5-0950-4e8a-859f-80a69047e424 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:07.750142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704afaa4-3190-4cc5-9ef4-4b93626a2b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:07.750819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704afaa4-3190-4cc5-9ef4-4b93626a2b34 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:07.751091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704afaa4-3190-4cc5-9ef4-4b93626a2b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:07.751416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704afaa4-3190-4cc5-9ef4-4b93626a2b34 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:07.752156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704afaa4-3190-4cc5-9ef4-4b93626a2b34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:07.752486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704afaa4-3190-4cc5-9ef4-4b93626a2b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:07.754021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704afaa4-3190-4cc5-9ef4-4b93626a2b34 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:07.759495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd3fb8-843b-4431-bf4f-63ddc1286fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:07.760177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3fb8-843b-4431-bf4f-63ddc1286fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:07.760486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3fb8-843b-4431-bf4f-63ddc1286fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:07.760767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3fb8-843b-4431-bf4f-63ddc1286fc8 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:07.761603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd3fb8-843b-4431-bf4f-63ddc1286fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:07.761915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd3fb8-843b-4431-bf4f-63ddc1286fc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:07.767789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 318/647] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:07.767927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd3fb8-843b-4431-bf4f-63ddc1286fc8 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:07.768005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704afaa4-3190-4cc5-9ef4-4b93626a2b34 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:07.787263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 330/648] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:07.788374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd3fb8-843b-4431-bf4f-63ddc1286fc8 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:07.795513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:07.883075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704afaa4-3190-4cc5-9ef4-4b93626a2b34 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:07.895813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd3fb8-843b-4431-bf4f-63ddc1286fc8 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:07.924460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169f855-ad00-4bfd-9c09-7eaf425f17ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:07.925270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169f855-ad00-4bfd-9c09-7eaf425f17ab None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:07.925554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169f855-ad00-4bfd-9c09-7eaf425f17ab None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:07.925942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169f855-ad00-4bfd-9c09-7eaf425f17ab None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:07.926689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169f855-ad00-4bfd-9c09-7eaf425f17ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:07.926996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169f855-ad00-4bfd-9c09-7eaf425f17ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:07.928568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169f855-ad00-4bfd-9c09-7eaf425f17ab None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:07.938083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20ebc7-2b45-4e2a-9c24-3b10a2093322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:07.938907 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20ebc7-2b45-4e2a-9c24-3b10a2093322 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:07.939196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20ebc7-2b45-4e2a-9c24-3b10a2093322 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:07.939471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20ebc7-2b45-4e2a-9c24-3b10a2093322 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:07.943921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 319/649] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:07.945254 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169f855-ad00-4bfd-9c09-7eaf425f17ab None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:07.949829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20ebc7-2b45-4e2a-9c24-3b10a2093322 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:07.951309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20ebc7-2b45-4e2a-9c24-3b10a2093322 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:07.953499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-fd20ebc7-2b45-4e2a-9c24-3b10a2093322 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:07.968663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:07.969854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 331/650] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:07.970890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20ebc7-2b45-4e2a-9c24-3b10a2093322 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:08.042921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:08.066692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169f855-ad00-4bfd-9c09-7eaf425f17ab None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:08.088020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20ebc7-2b45-4e2a-9c24-3b10a2093322 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:08.106567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eed578-8077-4255-bd97-55bbae4ac71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:08.107366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed578-8077-4255-bd97-55bbae4ac71b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.107647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed578-8077-4255-bd97-55bbae4ac71b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.107923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed578-8077-4255-bd97-55bbae4ac71b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.119042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eed578-8077-4255-bd97-55bbae4ac71b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:08.120522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eed578-8077-4255-bd97-55bbae4ac71b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:08.124857 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ed736-f728-4efd-bbec-8c3dbf898e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:08.125574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ed736-f728-4efd-bbec-8c3dbf898e13 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.125923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ed736-f728-4efd-bbec-8c3dbf898e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.126262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ed736-f728-4efd-bbec-8c3dbf898e13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b6eed578-8077-4255-bd97-55bbae4ac71b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:08.139335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:08.140275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:08.141272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:08.141272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:08.141272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:08.141272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:08.141272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 320/651] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:08.141664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eed578-8077-4255-bd97-55bbae4ac71b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:08.197094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:08.204304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ed736-f728-4efd-bbec-8c3dbf898e13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:08.205805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ed736-f728-4efd-bbec-8c3dbf898e13 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:08.252756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eed578-8077-4255-bd97-55bbae4ac71b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:08.284662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:08.291735 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef3153-d4f3-4993-9180-793b3774ae3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:08.292572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef3153-d4f3-4993-9180-793b3774ae3f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.292865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef3153-d4f3-4993-9180-793b3774ae3f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.293192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef3153-d4f3-4993-9180-793b3774ae3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.299962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef3153-d4f3-4993-9180-793b3774ae3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:08.301453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef3153-d4f3-4993-9180-793b3774ae3f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:08.312114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 332/652] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:52:08.312998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ed736-f728-4efd-bbec-8c3dbf898e13 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b1ef3153-d4f3-4993-9180-793b3774ae3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:08.315221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:08.316328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:08.317642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:08.317642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:08.317642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:08.317642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:08.317642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:08.317642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:08.317642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:08.317642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:08.317642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 321/653] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:08.317642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef3153-d4f3-4993-9180-793b3774ae3f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:08.412485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:08.417317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ed736-f728-4efd-bbec-8c3dbf898e13 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:08.421778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef3153-d4f3-4993-9180-793b3774ae3f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:08.451739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3827de-091c-4fe0-91b0-52cf838df1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:08.452421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3827de-091c-4fe0-91b0-52cf838df1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.452703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3827de-091c-4fe0-91b0-52cf838df1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.452974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3827de-091c-4fe0-91b0-52cf838df1d0 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.453794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3827de-091c-4fe0-91b0-52cf838df1d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:08.454100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3827de-091c-4fe0-91b0-52cf838df1d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:08.455516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3827de-091c-4fe0-91b0-52cf838df1d0 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:08.456051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ad655-d49f-4d6d-b4e3-2ef45b36af3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:08.456742 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad655-d49f-4d6d-b4e3-2ef45b36af3b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.457112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad655-d49f-4d6d-b4e3-2ef45b36af3b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.457293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad655-d49f-4d6d-b4e3-2ef45b36af3b None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.458006 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ad655-d49f-4d6d-b4e3-2ef45b36af3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:08.458305 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ad655-d49f-4d6d-b4e3-2ef45b36af3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:08.459655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ad655-d49f-4d6d-b4e3-2ef45b36af3b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:08.472034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 322/654] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:08.477480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998990d-6b33-4c2c-accd-e85b3c5d1063 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.477658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 333/655] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Jun 8 01:52:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:08.477754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998990d-6b33-4c2c-accd-e85b3c5d1063 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.477754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998990d-6b33-4c2c-accd-e85b3c5d1063 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.477914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3827de-091c-4fe0-91b0-52cf838df1d0 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:08.477995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 323/656] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:08.477995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998990d-6b33-4c2c-accd-e85b3c5d1063 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:08.531201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:08.578889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3827de-091c-4fe0-91b0-52cf838df1d0 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:08.579390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998990d-6b33-4c2c-accd-e85b3c5d1063 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:08.612260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90ffff-3854-4876-96c8-0d890723ad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:08.612466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2586d77-ce9c-48cb-88d6-a87022f731b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:08.612957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2586d77-ce9c-48cb-88d6-a87022f731b8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.613082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90ffff-3854-4876-96c8-0d890723ad2d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.613241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2586d77-ce9c-48cb-88d6-a87022f731b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.613368 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90ffff-3854-4876-96c8-0d890723ad2d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.613517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2586d77-ce9c-48cb-88d6-a87022f731b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.613646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90ffff-3854-4876-96c8-0d890723ad2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.614295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2586d77-ce9c-48cb-88d6-a87022f731b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:08.614605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2586d77-ce9c-48cb-88d6-a87022f731b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:08.616051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2586d77-ce9c-48cb-88d6-a87022f731b8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:08.624306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90ffff-3854-4876-96c8-0d890723ad2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:08.625880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90ffff-3854-4876-96c8-0d890723ad2d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:08.628933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 324/657] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:08.629875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2586d77-ce9c-48cb-88d6-a87022f731b8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-6e90ffff-3854-4876-96c8-0d890723ad2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:08.644221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:08.645319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:08.651314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:08.651314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:08.651314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:08.651314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:08.651314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 334/658] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:08.651314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90ffff-3854-4876-96c8-0d890723ad2d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:08.658212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:08.732367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2586d77-ce9c-48cb-88d6-a87022f731b8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:08.744716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:08.748224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90ffff-3854-4876-96c8-0d890723ad2d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:08.767262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d18fc5-f795-4ed3-85cf-324310efd70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:08.768063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d18fc5-f795-4ed3-85cf-324310efd70b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.768347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d18fc5-f795-4ed3-85cf-324310efd70b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.768653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d18fc5-f795-4ed3-85cf-324310efd70b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.776736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d18fc5-f795-4ed3-85cf-324310efd70b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:08.778182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d18fc5-f795-4ed3-85cf-324310efd70b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:08.782480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d0079-2f66-4cb9-b74d-0afb9bf45e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:08.783120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d0079-2f66-4cb9-b74d-0afb9bf45e85 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.783404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d0079-2f66-4cb9-b74d-0afb9bf45e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.783682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d0079-2f66-4cb9-b74d-0afb9bf45e85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-17d18fc5-f795-4ed3-85cf-324310efd70b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:08.793115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:08.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:08.795196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 325/659] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:08.795902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3c13f-3b14-46a1-87db-e2e69b3a2c39 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:08.796190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3c13f-3b14-46a1-87db-e2e69b3a2c39 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:08.796493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3c13f-3b14-46a1-87db-e2e69b3a2c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:08.812599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-05a3c13f-3b14-46a1-87db-e2e69b3a2c39 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:08.813818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-05a3c13f-3b14-46a1-87db-e2e69b3a2c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:08.856040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d0079-2f66-4cb9-b74d-0afb9bf45e85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:08.857448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d0079-2f66-4cb9-b74d-0afb9bf45e85 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:08.867397 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:08.880077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 326/660] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:08.881017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3c13f-3b14-46a1-87db-e2e69b3a2c39 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:08.971526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 335/661] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:52:08.972471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d0079-2f66-4cb9-b74d-0afb9bf45e85 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:08.982465 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3c13f-3b14-46a1-87db-e2e69b3a2c39 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:08.984017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:09.016625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3096b0-fe50-4718-b387-bcf63b38ee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:09.017286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3096b0-fe50-4718-b387-bcf63b38ee6b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:09.017569 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3096b0-fe50-4718-b387-bcf63b38ee6b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:09.017844 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3096b0-fe50-4718-b387-bcf63b38ee6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:09.018665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3096b0-fe50-4718-b387-bcf63b38ee6b None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:09.018979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3096b0-fe50-4718-b387-bcf63b38ee6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:09.020436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3096b0-fe50-4718-b387-bcf63b38ee6b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:09.026732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 327/662] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:08 2022] GET /identity/v3/role_assignments?user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:09.027503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3096b0-fe50-4718-b387-bcf63b38ee6b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:09.089063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d0079-2f66-4cb9-b74d-0afb9bf45e85 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:09.101291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:09.130645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda250fa-cc8e-4ddc-b316-527915a45ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:09.131248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda250fa-cc8e-4ddc-b316-527915a45ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:09.131579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda250fa-cc8e-4ddc-b316-527915a45ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:09.131862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda250fa-cc8e-4ddc-b316-527915a45ab0 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:09.133396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda250fa-cc8e-4ddc-b316-527915a45ab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:09.133721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda250fa-cc8e-4ddc-b316-527915a45ab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:09.135288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda250fa-cc8e-4ddc-b316-527915a45ab0 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:09.140580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3096b0-fe50-4718-b387-bcf63b38ee6b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:09.153115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 336/663] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:09.153115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda250fa-cc8e-4ddc-b316-527915a45ab0 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:09.183231 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02acff34-7aff-42d2-9483-6e7f2b9bf888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:09.183987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acff34-7aff-42d2-9483-6e7f2b9bf888 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:09.184364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acff34-7aff-42d2-9483-6e7f2b9bf888 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:09.184774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acff34-7aff-42d2-9483-6e7f2b9bf888 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:09.185628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02acff34-7aff-42d2-9483-6e7f2b9bf888 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:09.185944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02acff34-7aff-42d2-9483-6e7f2b9bf888 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:09.187447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02acff34-7aff-42d2-9483-6e7f2b9bf888 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:09.207932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 328/664] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Jun 8 01:52:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:09.207932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02acff34-7aff-42d2-9483-6e7f2b9bf888 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:09.241352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:09.276991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda250fa-cc8e-4ddc-b316-527915a45ab0 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:09.318997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759072b-3d4f-47b3-a75d-9f275c87fc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:09.319920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759072b-3d4f-47b3-a75d-9f275c87fc5e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:09.320202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759072b-3d4f-47b3-a75d-9f275c87fc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:09.320502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759072b-3d4f-47b3-a75d-9f275c87fc5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:09.332174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759072b-3d4f-47b3-a75d-9f275c87fc5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:09.333606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02acff34-7aff-42d2-9483-6e7f2b9bf888 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c759072b-3d4f-47b3-a75d-9f275c87fc5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:09.340879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:09.341842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 337/665] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:09.343649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759072b-3d4f-47b3-a75d-9f275c87fc5e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:09.375138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123a1a9-0903-4e09-b076-221d4a793c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:09.375943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123a1a9-0903-4e09-b076-221d4a793c10 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:09.376230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123a1a9-0903-4e09-b076-221d4a793c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:09.376544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123a1a9-0903-4e09-b076-221d4a793c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:09.377092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:09.387139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123a1a9-0903-4e09-b076-221d4a793c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:09.388581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123a1a9-0903-4e09-b076-221d4a793c10 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a123a1a9-0903-4e09-b076-221d4a793c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:09.406908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:09.407837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:09.408796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:09.408796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:09.408796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:09.408796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:09.408796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 329/666] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:09.409073 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123a1a9-0903-4e09-b076-221d4a793c10 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:09.482725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759072b-3d4f-47b3-a75d-9f275c87fc5e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:09.504467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:09.522477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef7226-219f-43e6-8b49-f80c8399fadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:09.523692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef7226-219f-43e6-8b49-f80c8399fadd None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:09.523810 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef7226-219f-43e6-8b49-f80c8399fadd None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:09.524044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef7226-219f-43e6-8b49-f80c8399fadd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:09.530380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123a1a9-0903-4e09-b076-221d4a793c10 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:09.531603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef7226-219f-43e6-8b49-f80c8399fadd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:09.533507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef7226-219f-43e6-8b49-f80c8399fadd None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-21ef7226-219f-43e6-8b49-f80c8399fadd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:09.548257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:09.549338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:09.550240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:09.550240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:09.550240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:09.550240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:09.550240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:09.550240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:09.550240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:09.550240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:09.550240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 338/667] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:09.550240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef7226-219f-43e6-8b49-f80c8399fadd None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:09.574149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03d9aa-3533-40a6-94e9-5f5b1781f98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:09.574816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03d9aa-3533-40a6-94e9-5f5b1781f98c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:09.575104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03d9aa-3533-40a6-94e9-5f5b1781f98c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:09.575387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03d9aa-3533-40a6-94e9-5f5b1781f98c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:09.647645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03d9aa-3533-40a6-94e9-5f5b1781f98c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:09.649189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03d9aa-3533-40a6-94e9-5f5b1781f98c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:09.649841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:09.664967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef7226-219f-43e6-8b49-f80c8399fadd None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:09.702995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b1fde-2a58-45ee-a292-3c786bfc404c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:09.703662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b1fde-2a58-45ee-a292-3c786bfc404c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:09.703943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b1fde-2a58-45ee-a292-3c786bfc404c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:09.704214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b1fde-2a58-45ee-a292-3c786bfc404c None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:09.705141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b1fde-2a58-45ee-a292-3c786bfc404c None admin] RBAC: Adding query filter params (name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:09.705444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b1fde-2a58-45ee-a292-3c786bfc404c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:09.706813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b1fde-2a58-45ee-a292-3c786bfc404c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:09.720543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 339/668] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:52:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:09.720543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b1fde-2a58-45ee-a292-3c786bfc404c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:09.734128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:09.770567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 330/669] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:52:09.770567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03d9aa-3533-40a6-94e9-5f5b1781f98c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:09.833775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b1fde-2a58-45ee-a292-3c786bfc404c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:09.870999 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adafa85-3aa2-4c93-a801-776e1655f1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:09.871668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adafa85-3aa2-4c93-a801-776e1655f1ed None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:09.871948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adafa85-3aa2-4c93-a801-776e1655f1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:09.872226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adafa85-3aa2-4c93-a801-776e1655f1ed None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:09.872976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adafa85-3aa2-4c93-a801-776e1655f1ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:09.873284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adafa85-3aa2-4c93-a801-776e1655f1ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:09.874650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adafa85-3aa2-4c93-a801-776e1655f1ed None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:09.880363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03d9aa-3533-40a6-94e9-5f5b1781f98c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:09.887534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 340/670] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:09.888360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adafa85-3aa2-4c93-a801-776e1655f1ed None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:09.901360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:09.924354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea91d03-34a3-4f2b-a6e0-57e65132da45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:09.925058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea91d03-34a3-4f2b-a6e0-57e65132da45 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:09.925346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea91d03-34a3-4f2b-a6e0-57e65132da45 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:09.925644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea91d03-34a3-4f2b-a6e0-57e65132da45 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:09.926415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea91d03-34a3-4f2b-a6e0-57e65132da45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:09.926727 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea91d03-34a3-4f2b-a6e0-57e65132da45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:09.928182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea91d03-34a3-4f2b-a6e0-57e65132da45 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:09.946092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 331/671] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:09.946092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea91d03-34a3-4f2b-a6e0-57e65132da45 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:09.984704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:10.003601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adafa85-3aa2-4c93-a801-776e1655f1ed None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:10.040212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189cb816-b9a1-4604-883e-8eec3d0e7330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:10.041335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cb816-b9a1-4604-883e-8eec3d0e7330 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:10.041850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cb816-b9a1-4604-883e-8eec3d0e7330 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:10.042324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cb816-b9a1-4604-883e-8eec3d0e7330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:10.053271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189cb816-b9a1-4604-883e-8eec3d0e7330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:10.055884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189cb816-b9a1-4604-883e-8eec3d0e7330 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:10.061243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea91d03-34a3-4f2b-a6e0-57e65132da45 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-189cb816-b9a1-4604-883e-8eec3d0e7330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:10.074910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:10.075992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:10.077184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:10.077184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:10.077184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:10.077184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:10.077981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 341/672] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:10.082865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189cb816-b9a1-4604-883e-8eec3d0e7330 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:10.101278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc02c4d-ae0e-4da6-9b09-7a1959f870ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:10.102206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc02c4d-ae0e-4da6-9b09-7a1959f870ae None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:10.102486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc02c4d-ae0e-4da6-9b09-7a1959f870ae None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:10.102761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc02c4d-ae0e-4da6-9b09-7a1959f870ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:10.114263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc02c4d-ae0e-4da6-9b09-7a1959f870ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:10.116955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc02c4d-ae0e-4da6-9b09-7a1959f870ae None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-cdc02c4d-ae0e-4da6-9b09-7a1959f870ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:10.136534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:10.137601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:10.143563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:10.143563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:10.143563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:10.143563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:10.143563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 332/673] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:10.143563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc02c4d-ae0e-4da6-9b09-7a1959f870ae None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:10.151262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:10.195383 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189cb816-b9a1-4604-883e-8eec3d0e7330 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:10.235159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486848b-97bd-43b9-af58-c5bc5bf65a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:10.235959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486848b-97bd-43b9-af58-c5bc5bf65a27 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:10.236245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486848b-97bd-43b9-af58-c5bc5bf65a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:10.236559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486848b-97bd-43b9-af58-c5bc5bf65a27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:10.237468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:10.245131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486848b-97bd-43b9-af58-c5bc5bf65a27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:10.246630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486848b-97bd-43b9-af58-c5bc5bf65a27 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:10.252311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc02c4d-ae0e-4da6-9b09-7a1959f870ae None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-4486848b-97bd-43b9-af58-c5bc5bf65a27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:10.261218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:10.262129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:10.263029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:10.263029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:10.263029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:10.263029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:10.263029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:10.263029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:10.263029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:10.263029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:10.263029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 342/674] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:10.263515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486848b-97bd-43b9-af58-c5bc5bf65a27 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:10.295222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c01ef-2cc1-4257-80e0-0abab93de116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:10.295884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c01ef-2cc1-4257-80e0-0abab93de116 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:10.296169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c01ef-2cc1-4257-80e0-0abab93de116 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:10.296540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c01ef-2cc1-4257-80e0-0abab93de116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:10.297367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c01ef-2cc1-4257-80e0-0abab93de116 None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=236226ec865b49febc7713df1c0fab98) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:10.297680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c01ef-2cc1-4257-80e0-0abab93de116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:10.299121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c01ef-2cc1-4257-80e0-0abab93de116 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:10.316431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 333/675] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:10 2022] GET /identity/v3/role_assignments?user.id=236226ec865b49febc7713df1c0fab98&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:10.316431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c01ef-2cc1-4257-80e0-0abab93de116 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:10.389474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486848b-97bd-43b9-af58-c5bc5bf65a27 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:10.420230 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:10.435376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1e1a1-48d9-4b51-a598-5b189a01eb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:10.436156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1e1a1-48d9-4b51-a598-5b189a01eb0a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:10.436773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1e1a1-48d9-4b51-a598-5b189a01eb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:10.436895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1e1a1-48d9-4b51-a598-5b189a01eb0a None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:10.437731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1e1a1-48d9-4b51-a598-5b189a01eb0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:10.438090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1e1a1-48d9-4b51-a598-5b189a01eb0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:10.439653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1e1a1-48d9-4b51-a598-5b189a01eb0a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:10.440489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c01ef-2cc1-4257-80e0-0abab93de116 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:10.456485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 343/676] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:10.457659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1e1a1-48d9-4b51-a598-5b189a01eb0a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:10.487018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b6407-e9c7-4ea0-b5e0-0ea199596ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:10.487852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b6407-e9c7-4ea0-b5e0-0ea199596ffb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:10.488207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b6407-e9c7-4ea0-b5e0-0ea199596ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:10.488583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b6407-e9c7-4ea0-b5e0-0ea199596ffb None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:10.489882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b6407-e9c7-4ea0-b5e0-0ea199596ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:10.489882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b6407-e9c7-4ea0-b5e0-0ea199596ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:10.491542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b6407-e9c7-4ea0-b5e0-0ea199596ffb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:10.511466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 334/677] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:10.512709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b6407-e9c7-4ea0-b5e0-0ea199596ffb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:10.530880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:10.599867 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1e1a1-48d9-4b51-a598-5b189a01eb0a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:10.642598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b6407-e9c7-4ea0-b5e0-0ea199596ffb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:10.643988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f836e-f625-4e10-bce3-8485117f05a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:10.645012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f836e-f625-4e10-bce3-8485117f05a4 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:10.645348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f836e-f625-4e10-bce3-8485117f05a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:10.645639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f836e-f625-4e10-bce3-8485117f05a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:10.647414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f836e-f625-4e10-bce3-8485117f05a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:10.647796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f836e-f625-4e10-bce3-8485117f05a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:10.649411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03f836e-f625-4e10-bce3-8485117f05a4 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:10.666790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 344/678] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:10.667637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f836e-f625-4e10-bce3-8485117f05a4 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:10.693130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69394372-f3d8-4041-96e8-27729372c335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:10.693945 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69394372-f3d8-4041-96e8-27729372c335 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:10.694236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69394372-f3d8-4041-96e8-27729372c335 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:10.695049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69394372-f3d8-4041-96e8-27729372c335 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:10.705916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69394372-f3d8-4041-96e8-27729372c335 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:10.707436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69394372-f3d8-4041-96e8-27729372c335 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:10.718610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-69394372-f3d8-4041-96e8-27729372c335 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:10.728542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:10.729487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:10.730438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 335/679] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:10.731048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69394372-f3d8-4041-96e8-27729372c335 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:10.799286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f836e-f625-4e10-bce3-8485117f05a4 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:10.813037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:10.847070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69394372-f3d8-4041-96e8-27729372c335 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:10.848387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655c7e6-cbc1-4fda-925e-6b1fac027c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:10.849475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655c7e6-cbc1-4fda-925e-6b1fac027c9d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:10.849766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655c7e6-cbc1-4fda-925e-6b1fac027c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:10.850047 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655c7e6-cbc1-4fda-925e-6b1fac027c9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:10.859127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655c7e6-cbc1-4fda-925e-6b1fac027c9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:10.861124 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655c7e6-cbc1-4fda-925e-6b1fac027c9d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-5655c7e6-cbc1-4fda-925e-6b1fac027c9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:10.880541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:10.881496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:10.882656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:10.882656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:10.882656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:10.882656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:10.882656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:10.882656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:10.882656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:10.882656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:10.885451 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 345/680] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:10.886514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655c7e6-cbc1-4fda-925e-6b1fac027c9d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:10.890181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ff37-2c24-45d5-8108-4b59a55554d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:10.890855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ff37-2c24-45d5-8108-4b59a55554d9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:10.891142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ff37-2c24-45d5-8108-4b59a55554d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:10.891462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ff37-2c24-45d5-8108-4b59a55554d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:10.893345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ff37-2c24-45d5-8108-4b59a55554d9 None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=7d1e01f6f82a4ebbb15d0cb55b963af4) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:10.893668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ff37-2c24-45d5-8108-4b59a55554d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:10.895133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ff37-2c24-45d5-8108-4b59a55554d9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:10.908430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 336/681] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:10 2022] GET /identity/v3/role_assignments?user.id=7d1e01f6f82a4ebbb15d0cb55b963af4&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:10.908430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ff37-2c24-45d5-8108-4b59a55554d9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:11.040997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ff37-2c24-45d5-8108-4b59a55554d9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:11.088424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df11263-485f-4bb0-90fb-022ce31db589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:11.089110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df11263-485f-4bb0-90fb-022ce31db589 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:11.089398 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df11263-485f-4bb0-90fb-022ce31db589 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:11.089677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df11263-485f-4bb0-90fb-022ce31db589 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:11.096420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df11263-485f-4bb0-90fb-022ce31db589 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:11.096420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df11263-485f-4bb0-90fb-022ce31db589 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:11.096420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df11263-485f-4bb0-90fb-022ce31db589 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:11.096420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:11.109515 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 337/682] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:11.117619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df11263-485f-4bb0-90fb-022ce31db589 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:11.125280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655c7e6-cbc1-4fda-925e-6b1fac027c9d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:11.127181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:11.174899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eb7e0-bfd7-437a-8213-f8e27e3e8269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:11.175972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eb7e0-bfd7-437a-8213-f8e27e3e8269 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:11.176485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eb7e0-bfd7-437a-8213-f8e27e3e8269 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:11.178756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eb7e0-bfd7-437a-8213-f8e27e3e8269 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:11.180524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eb7e0-bfd7-437a-8213-f8e27e3e8269 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:11.184699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eb7e0-bfd7-437a-8213-f8e27e3e8269 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:11.186470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eb7e0-bfd7-437a-8213-f8e27e3e8269 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:11.221701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 346/683] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jun 8 01:52:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:11.227422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eb7e0-bfd7-437a-8213-f8e27e3e8269 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:11.276249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df11263-485f-4bb0-90fb-022ce31db589 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:11.314246 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4fcdfb-beb6-4365-ab77-bfc3822bbe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:11.314843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fcdfb-beb6-4365-ab77-bfc3822bbe0d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:11.315130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fcdfb-beb6-4365-ab77-bfc3822bbe0d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:11.315410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4fcdfb-beb6-4365-ab77-bfc3822bbe0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:11.316241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fcdfb-beb6-4365-ab77-bfc3822bbe0d None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=d26f4426ba8c43399c4a7ef2f1701f57) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:11.316588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fcdfb-beb6-4365-ab77-bfc3822bbe0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:11.318170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4fcdfb-beb6-4365-ab77-bfc3822bbe0d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:11.325689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 338/684] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:11 2022] GET /identity/v3/role_assignments?user.id=d26f4426ba8c43399c4a7ef2f1701f57&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:11.349140 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eb7e0-bfd7-437a-8213-f8e27e3e8269 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:11.390280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958c084-6164-4d48-b7f8-07abd5967abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:11.390956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958c084-6164-4d48-b7f8-07abd5967abd None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:11.391244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958c084-6164-4d48-b7f8-07abd5967abd None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:11.391526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958c084-6164-4d48-b7f8-07abd5967abd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:11.405397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e8e06-26e6-4744-97e2-f68ee9bd1809 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:11.405686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e8e06-26e6-4744-97e2-f68ee9bd1809 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:11.405968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e8e06-26e6-4744-97e2-f68ee9bd1809 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:11.407376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 339/685] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:11.414350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62170cdb-0f78-4dcb-a137-a1a7021b64c7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:11.414781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62170cdb-0f78-4dcb-a137-a1a7021b64c7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:11.415279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62170cdb-0f78-4dcb-a137-a1a7021b64c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:11.433121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:11.436653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-62170cdb-0f78-4dcb-a137-a1a7021b64c7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:11.438028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-62170cdb-0f78-4dcb-a137-a1a7021b64c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:11.488704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958c084-6164-4d48-b7f8-07abd5967abd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:11.490584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958c084-6164-4d48-b7f8-07abd5967abd None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:11.519973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 340/686] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:52:11.520790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:11.545746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62170cdb-0f78-4dcb-a137-a1a7021b64c7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:11.620923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 347/687] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 08 01:52:11.629556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958c084-6164-4d48-b7f8-07abd5967abd None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:11.666128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62170cdb-0f78-4dcb-a137-a1a7021b64c7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:11.709518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc4254-2d03-4af8-989a-941aae9fc1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:11.710349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4254-2d03-4af8-989a-941aae9fc1ac None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:11.710841 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4254-2d03-4af8-989a-941aae9fc1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:11.711261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4254-2d03-4af8-989a-941aae9fc1ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:11.714428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:11.725722 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc4254-2d03-4af8-989a-941aae9fc1ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-37fc4254-2d03-4af8-989a-941aae9fc1ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:11.733991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:11.735166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:11.736252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 341/688] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:11.743195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc4254-2d03-4af8-989a-941aae9fc1ac None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:11.772384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958c084-6164-4d48-b7f8-07abd5967abd None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:11.812807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:11.814071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67778-2d2b-4f03-9429-78bd7e587436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:11.814879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67778-2d2b-4f03-9429-78bd7e587436 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:11.815284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67778-2d2b-4f03-9429-78bd7e587436 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:11.815560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67778-2d2b-4f03-9429-78bd7e587436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:11.827320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67778-2d2b-4f03-9429-78bd7e587436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:11.828969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67778-2d2b-4f03-9429-78bd7e587436 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-60b67778-2d2b-4f03-9429-78bd7e587436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:11.848215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:11.849388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:11.850703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:11.850703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:11.850703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:11.850703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:11.850703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 348/689] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:11.855622 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67778-2d2b-4f03-9429-78bd7e587436 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:11.877825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc4254-2d03-4af8-989a-941aae9fc1ac None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:11.922378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceb66d-e96c-41b9-94ca-bd55d9f1e5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:11.923059 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb66d-e96c-41b9-94ca-bd55d9f1e5de None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:11.923339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb66d-e96c-41b9-94ca-bd55d9f1e5de None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:11.923609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb66d-e96c-41b9-94ca-bd55d9f1e5de None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:11.924827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceb66d-e96c-41b9-94ca-bd55d9f1e5de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:11.925144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceb66d-e96c-41b9-94ca-bd55d9f1e5de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:11.926785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ceb66d-e96c-41b9-94ca-bd55d9f1e5de None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:11.938694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 342/690] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:11.944371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceb66d-e96c-41b9-94ca-bd55d9f1e5de None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:11.979160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67778-2d2b-4f03-9429-78bd7e587436 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:11.993506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:12.017861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db257d-0649-4a6b-a7ae-15b449db1e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:12.018618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db257d-0649-4a6b-a7ae-15b449db1e5a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:12.018991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db257d-0649-4a6b-a7ae-15b449db1e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:12.019339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db257d-0649-4a6b-a7ae-15b449db1e5a None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:12.020232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db257d-0649-4a6b-a7ae-15b449db1e5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:12.020729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db257d-0649-4a6b-a7ae-15b449db1e5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:12.022179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db257d-0649-4a6b-a7ae-15b449db1e5a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:12.035402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 349/691] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:12.041727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db257d-0649-4a6b-a7ae-15b449db1e5a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:12.075482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ceb66d-e96c-41b9-94ca-bd55d9f1e5de None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:12.094382 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:12.114706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011335d-6e91-437a-a23f-a6a5c36bfc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:12.115777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011335d-6e91-437a-a23f-a6a5c36bfc06 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:12.116292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011335d-6e91-437a-a23f-a6a5c36bfc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:12.116735 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011335d-6e91-437a-a23f-a6a5c36bfc06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:12.128894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011335d-6e91-437a-a23f-a6a5c36bfc06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:12.130697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011335d-6e91-437a-a23f-a6a5c36bfc06 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d011335d-6e91-437a-a23f-a6a5c36bfc06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:12.151226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:12.152709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:12.155113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:12.155113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:12.155113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:12.155113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:12.155113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 343/692] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:12.159327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011335d-6e91-437a-a23f-a6a5c36bfc06 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:12.271550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db257d-0649-4a6b-a7ae-15b449db1e5a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:12.310329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec0b8f-9178-4298-b01c-93ff28520c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:12.311125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec0b8f-9178-4298-b01c-93ff28520c80 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:12.311404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec0b8f-9178-4298-b01c-93ff28520c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:12.311676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec0b8f-9178-4298-b01c-93ff28520c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:12.315184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011335d-6e91-437a-a23f-a6a5c36bfc06 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:12.319291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec0b8f-9178-4298-b01c-93ff28520c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:12.322366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec0b8f-9178-4298-b01c-93ff28520c80 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-ffec0b8f-9178-4298-b01c-93ff28520c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:12.340645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:12.342475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:12.343561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:12.343561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:12.343561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:12.343561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:12.343561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:12.343561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:12.343561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:12.343561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:12.343561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 350/693] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:12.349378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec0b8f-9178-4298-b01c-93ff28520c80 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:12.389175 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cc0d7-009b-4a04-81c9-fcb2ae7c616b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:12.389936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cc0d7-009b-4a04-81c9-fcb2ae7c616b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:12.390147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:12.390319 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cc0d7-009b-4a04-81c9-fcb2ae7c616b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:12.390647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cc0d7-009b-4a04-81c9-fcb2ae7c616b None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:12.391447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cc0d7-009b-4a04-81c9-fcb2ae7c616b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:12.391854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cc0d7-009b-4a04-81c9-fcb2ae7c616b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:12.393349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cc0d7-009b-4a04-81c9-fcb2ae7c616b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:12.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 344/694] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:12.430164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cc0d7-009b-4a04-81c9-fcb2ae7c616b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:12.475559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec0b8f-9178-4298-b01c-93ff28520c80 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:12.488725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:12.516807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc2168-d4ee-4f11-adde-20837f8d3b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:12.517477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc2168-d4ee-4f11-adde-20837f8d3b73 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:12.517758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc2168-d4ee-4f11-adde-20837f8d3b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:12.518030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc2168-d4ee-4f11-adde-20837f8d3b73 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:12.518818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc2168-d4ee-4f11-adde-20837f8d3b73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:12.519126 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc2168-d4ee-4f11-adde-20837f8d3b73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:12.520990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc2168-d4ee-4f11-adde-20837f8d3b73 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:12.535046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 351/695] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:12.542218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc2168-d4ee-4f11-adde-20837f8d3b73 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:12.645268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cc0d7-009b-4a04-81c9-fcb2ae7c616b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:12.711053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:12.720758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc2168-d4ee-4f11-adde-20837f8d3b73 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:12.722608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e884109-782c-4848-bfa4-bde49adf9f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:12.723733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884109-782c-4848-bfa4-bde49adf9f88 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:12.724105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884109-782c-4848-bfa4-bde49adf9f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:12.724383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884109-782c-4848-bfa4-bde49adf9f88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:12.741768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e884109-782c-4848-bfa4-bde49adf9f88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:12.743202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e884109-782c-4848-bfa4-bde49adf9f88 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-6e884109-782c-4848-bfa4-bde49adf9f88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:12.767885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:12.768916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:12.769869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 345/696] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jun 8 01:52:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:12.777667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e884109-782c-4848-bfa4-bde49adf9f88 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:12.778007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060be39d-4199-44f0-a86f-bcee40e4698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:12.778688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be39d-4199-44f0-a86f-bcee40e4698b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:12.778975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be39d-4199-44f0-a86f-bcee40e4698b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:12.779261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be39d-4199-44f0-a86f-bcee40e4698b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:12.780106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060be39d-4199-44f0-a86f-bcee40e4698b None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=fc299258fe814636880cf595fc2904b4) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:12.780564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060be39d-4199-44f0-a86f-bcee40e4698b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:12.782117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060be39d-4199-44f0-a86f-bcee40e4698b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:12.802768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 352/697] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:12 2022] GET /identity/v3/role_assignments?user.id=fc299258fe814636880cf595fc2904b4&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:12.876773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:12.888265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e884109-782c-4848-bfa4-bde49adf9f88 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:12.926023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007168b-a1c1-4ac9-8a33-8d53bb3a206f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:12.926718 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007168b-a1c1-4ac9-8a33-8d53bb3a206f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:12.927049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007168b-a1c1-4ac9-8a33-8d53bb3a206f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:12.927329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007168b-a1c1-4ac9-8a33-8d53bb3a206f None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:12.928184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007168b-a1c1-4ac9-8a33-8d53bb3a206f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:12.928526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007168b-a1c1-4ac9-8a33-8d53bb3a206f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:12.930125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007168b-a1c1-4ac9-8a33-8d53bb3a206f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:12.950829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 346/698] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jun 8 01:52:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:12.956415 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060be39d-4199-44f0-a86f-bcee40e4698b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:13.002484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677e78a-d714-4295-be67-acbfacfcb119 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:13.002765 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677e78a-d714-4295-be67-acbfacfcb119 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:13.003050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677e78a-d714-4295-be67-acbfacfcb119 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:13.008439 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 347/699] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:13.010756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2a100-38d5-43f5-8514-fd6645e4cf6c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:13.011052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2a100-38d5-43f5-8514-fd6645e4cf6c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:13.011331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2a100-38d5-43f5-8514-fd6645e4cf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:13.045511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6cb2a100-38d5-43f5-8514-fd6645e4cf6c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:13.046892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6cb2a100-38d5-43f5-8514-fd6645e4cf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:13.098465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060be39d-4199-44f0-a86f-bcee40e4698b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:13.151658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eaeb2e-fcb5-428b-8566-5520073e3980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:13.152367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eaeb2e-fcb5-428b-8566-5520073e3980 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:13.152706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eaeb2e-fcb5-428b-8566-5520073e3980 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:13.153846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eaeb2e-fcb5-428b-8566-5520073e3980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:13.185568 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:13.188165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 348/700] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:13.216775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2a100-38d5-43f5-8514-fd6645e4cf6c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:13.239745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eaeb2e-fcb5-428b-8566-5520073e3980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:13.242155 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eaeb2e-fcb5-428b-8566-5520073e3980 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:13.311323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:13.356343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2a100-38d5-43f5-8514-fd6645e4cf6c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:13.360481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 353/701] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:52:13.366384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eaeb2e-fcb5-428b-8566-5520073e3980 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:13.411004 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d167-85cf-4e26-91ca-fd2872b85726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:13.412180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d167-85cf-4e26-91ca-fd2872b85726 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:13.412710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d167-85cf-4e26-91ca-fd2872b85726 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:13.413617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d167-85cf-4e26-91ca-fd2872b85726 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:13.414911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274d167-85cf-4e26-91ca-fd2872b85726 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:13.417098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274d167-85cf-4e26-91ca-fd2872b85726 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:13.437069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0274d167-85cf-4e26-91ca-fd2872b85726 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:13.443368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:13.444900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 08 01:52:13.446490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:13.447281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 349/702] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Jun 8 01:52:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:13.455265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d167-85cf-4e26-91ca-fd2872b85726 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:13.498123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eaeb2e-fcb5-428b-8566-5520073e3980 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:13.545971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1cab1-5331-4bb7-9966-305e924cb062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:13.547403 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cab1-5331-4bb7-9966-305e924cb062 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:13.547888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cab1-5331-4bb7-9966-305e924cb062 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:13.548691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cab1-5331-4bb7-9966-305e924cb062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:13.563929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1cab1-5331-4bb7-9966-305e924cb062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:13.566295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1cab1-5331-4bb7-9966-305e924cb062 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-7da1cab1-5331-4bb7-9966-305e924cb062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:13.588180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:13.589186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:13.590076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:13.590076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:13.590076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:13.590076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:13.590076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 354/703] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:13.593335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d167-85cf-4e26-91ca-fd2872b85726 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:13.595254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1cab1-5331-4bb7-9966-305e924cb062 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:13.657109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a96490-17ea-4868-9a39-cec5568af205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:13.657802 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a96490-17ea-4868-9a39-cec5568af205 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:13.658093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a96490-17ea-4868-9a39-cec5568af205 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:13.658372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a96490-17ea-4868-9a39-cec5568af205 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:13.659093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a96490-17ea-4868-9a39-cec5568af205 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:13.659401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a96490-17ea-4868-9a39-cec5568af205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:13.661099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a96490-17ea-4868-9a39-cec5568af205 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:13.670525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:13.678797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 350/704] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jun 8 01:52:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:13.686157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a96490-17ea-4868-9a39-cec5568af205 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:13.716533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1cab1-5331-4bb7-9966-305e924cb062 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:13.722507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:13.751388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d040d50-9cc7-43cc-910d-4acd7c91abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:13.752330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d040d50-9cc7-43cc-910d-4acd7c91abfd None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:13.752776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d040d50-9cc7-43cc-910d-4acd7c91abfd None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:13.753209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d040d50-9cc7-43cc-910d-4acd7c91abfd None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:13.754067 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d040d50-9cc7-43cc-910d-4acd7c91abfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:13.754525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d040d50-9cc7-43cc-910d-4acd7c91abfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:13.756042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d040d50-9cc7-43cc-910d-4acd7c91abfd None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:13.769539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 355/705] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:13.775829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d040d50-9cc7-43cc-910d-4acd7c91abfd None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:13.891768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d040d50-9cc7-43cc-910d-4acd7c91abfd None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:13.931864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b1ee2-b25c-4ee1-b7fe-9a96fd722a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:13.933026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b1ee2-b25c-4ee1-b7fe-9a96fd722a59 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:13.933861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b1ee2-b25c-4ee1-b7fe-9a96fd722a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:13.934340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b1ee2-b25c-4ee1-b7fe-9a96fd722a59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:13.942135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b1ee2-b25c-4ee1-b7fe-9a96fd722a59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:13.943877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a96490-17ea-4868-9a39-cec5568af205 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:13.944578 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b1ee2-b25c-4ee1-b7fe-9a96fd722a59 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-ea0b1ee2-b25c-4ee1-b7fe-9a96fd722a59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:13.961650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:13.962614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:13.963579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:13.963579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:13.963579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:13.963579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:13.963579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:13.963579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:13.963579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:13.963579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:13.966482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 356/706] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:13.973074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cde4e9-7a82-44ba-a8e8-ad3a9971f733 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:13.973642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cde4e9-7a82-44ba-a8e8-ad3a9971f733 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:13.974345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cde4e9-7a82-44ba-a8e8-ad3a9971f733 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:13.977007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 357/707] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:13.978064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cde4e9-7a82-44ba-a8e8-ad3a9971f733 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:14.000279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10e7e9-548b-4a69-9f07-ce86ec8a98ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:14.001047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10e7e9-548b-4a69-9f07-ce86ec8a98ab None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:14.001339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10e7e9-548b-4a69-9f07-ce86ec8a98ab None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:14.001615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10e7e9-548b-4a69-9f07-ce86ec8a98ab None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:14.002418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10e7e9-548b-4a69-9f07-ce86ec8a98ab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:14.002732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10e7e9-548b-4a69-9f07-ce86ec8a98ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:14.004210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10e7e9-548b-4a69-9f07-ce86ec8a98ab None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:14.010617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 351/708] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jun 8 01:52:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:14.012062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1245d-d9df-45f2-921f-2238f25504b1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:14.012355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1245d-d9df-45f2-921f-2238f25504b1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:14.012670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1245d-d9df-45f2-921f-2238f25504b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:14.016001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:14.031532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-46c1245d-d9df-45f2-921f-2238f25504b1 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:14.039562 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-46c1245d-d9df-45f2-921f-2238f25504b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:14.165083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:14.174558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cde4e9-7a82-44ba-a8e8-ad3a9971f733 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:14.180285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 352/709] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:14.205485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1245d-d9df-45f2-921f-2238f25504b1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:14.245640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ebda2-95bb-4b80-86d0-4475c754debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:14.246526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ebda2-95bb-4b80-86d0-4475c754debc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:14.246809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ebda2-95bb-4b80-86d0-4475c754debc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:14.247084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ebda2-95bb-4b80-86d0-4475c754debc None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:14.247897 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ebda2-95bb-4b80-86d0-4475c754debc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:14.248207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ebda2-95bb-4b80-86d0-4475c754debc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:14.251156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ebda2-95bb-4b80-86d0-4475c754debc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:14.270193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 358/710] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:14.278583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ebda2-95bb-4b80-86d0-4475c754debc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:14.341529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1245d-d9df-45f2-921f-2238f25504b1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:14.385093 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:14.407293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95524ea-5852-414f-a8ec-0492db71d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:14.409502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95524ea-5852-414f-a8ec-0492db71d03d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:14.409615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95524ea-5852-414f-a8ec-0492db71d03d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:14.410065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95524ea-5852-414f-a8ec-0492db71d03d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:14.415046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ebda2-95bb-4b80-86d0-4475c754debc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:14.431461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95524ea-5852-414f-a8ec-0492db71d03d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:14.432969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95524ea-5852-414f-a8ec-0492db71d03d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a95524ea-5852-414f-a8ec-0492db71d03d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:14.466204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:14.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:14.468081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:14.468081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:14.468081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:14.468081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:14.473731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 353/711] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:14.477109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95524ea-5852-414f-a8ec-0492db71d03d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:14.486720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ece7bd-38b5-4e59-b8e2-3ac349809879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:14.487395 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ece7bd-38b5-4e59-b8e2-3ac349809879 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:14.487682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ece7bd-38b5-4e59-b8e2-3ac349809879 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:14.487962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ece7bd-38b5-4e59-b8e2-3ac349809879 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:14.488836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ece7bd-38b5-4e59-b8e2-3ac349809879 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:14.489157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ece7bd-38b5-4e59-b8e2-3ac349809879 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:14.493730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ece7bd-38b5-4e59-b8e2-3ac349809879 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:14.513425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 359/712] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:14 2022] GET /identity/v3/role_assignments?user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:14.557196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:14.568337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9ddc9-52b8-4164-8ebf-bbb20da85c1a None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:14.568704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9ddc9-52b8-4164-8ebf-bbb20da85c1a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:14.568926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9ddc9-52b8-4164-8ebf-bbb20da85c1a None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:14.571282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 360/713] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:14.585599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927026e1-da4f-41ca-94e1-2bf56464f878 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:14.585917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927026e1-da4f-41ca-94e1-2bf56464f878 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:14.586173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927026e1-da4f-41ca-94e1-2bf56464f878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:14.625422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-927026e1-da4f-41ca-94e1-2bf56464f878 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:14.627023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-927026e1-da4f-41ca-94e1-2bf56464f878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:14.681877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95524ea-5852-414f-a8ec-0492db71d03d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:14.731380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 361/714] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:14.758553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dc99b-fccc-450d-8e48-3289d3d9586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:14.759622 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dc99b-fccc-450d-8e48-3289d3d9586e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:14.760137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dc99b-fccc-450d-8e48-3289d3d9586e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:14.760679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dc99b-fccc-450d-8e48-3289d3d9586e None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:14.761659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dc99b-fccc-450d-8e48-3289d3d9586e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:14.762168 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dc99b-fccc-450d-8e48-3289d3d9586e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:14.763845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dc99b-fccc-450d-8e48-3289d3d9586e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:14.782808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 354/715] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:14.787399 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927026e1-da4f-41ca-94e1-2bf56464f878 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:14.791616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dc99b-fccc-450d-8e48-3289d3d9586e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:14.932755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:14.969766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927026e1-da4f-41ca-94e1-2bf56464f878 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:14.989258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dc99b-fccc-450d-8e48-3289d3d9586e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:14.997904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:15.034779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527853b7-3ee6-40fd-a4ba-0773cfbe4249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:15.035746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527853b7-3ee6-40fd-a4ba-0773cfbe4249 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:15.036036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527853b7-3ee6-40fd-a4ba-0773cfbe4249 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:15.036370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527853b7-3ee6-40fd-a4ba-0773cfbe4249 None admin] PATH_INFO: `/v3/services/network` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:15.037056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527853b7-3ee6-40fd-a4ba-0773cfbe4249 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:15.039016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527853b7-3ee6-40fd-a4ba-0773cfbe4249 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:15.055190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dcf96-5807-4518-ac29-46e71e525c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:15.056326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dcf96-5807-4518-ac29-46e71e525c75 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:15.056861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dcf96-5807-4518-ac29-46e71e525c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-527853b7-3ee6-40fd-a4ba-0773cfbe4249 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:15.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:15.058293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:15.059112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 362/716] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:14 2022] GET /identity/v3/services/network => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:15.060420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dcf96-5807-4518-ac29-46e71e525c75 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:15.061347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dcf96-5807-4518-ac29-46e71e525c75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:15.062997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dcf96-5807-4518-ac29-46e71e525c75 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:15.066083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527853b7-3ee6-40fd-a4ba-0773cfbe4249 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:15.100051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 355/717] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:52:14 2022] POST /identity/v3/users => generated 254 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:15.227299 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527853b7-3ee6-40fd-a4ba-0773cfbe4249 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:15.281402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd998292-8622-466c-8b40-fa23830804ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:15.282081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd998292-8622-466c-8b40-fa23830804ae None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:15.282370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd998292-8622-466c-8b40-fa23830804ae None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:15.282651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd998292-8622-466c-8b40-fa23830804ae None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:15.283371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd998292-8622-466c-8b40-fa23830804ae None admin] RBAC: Adding query filter params (name=network) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:15.283682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd998292-8622-466c-8b40-fa23830804ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:15.285194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd998292-8622-466c-8b40-fa23830804ae None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:15.294437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 363/718] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:15.299859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dcf96-5807-4518-ac29-46e71e525c75 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:15.458315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dcf96-5807-4518-ac29-46e71e525c75 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:15.496308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41410e-eed5-4d68-a42b-edb05801d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:15.497374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41410e-eed5-4d68-a42b-edb05801d1fa None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:15.497880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41410e-eed5-4d68-a42b-edb05801d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:15.498346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41410e-eed5-4d68-a42b-edb05801d1fa None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:15.499239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41410e-eed5-4d68-a42b-edb05801d1fa None admin] RBAC: Adding query filter params (type=network) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:15.499736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41410e-eed5-4d68-a42b-edb05801d1fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:15.501328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41410e-eed5-4d68-a42b-edb05801d1fa None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:15.506364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 356/719] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:15 2022] GET /identity/v3/services?type=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:15.908590 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb0267-cc63-4f9f-bdad-2110cc04a849 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:15.909100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb0267-cc63-4f9f-bdad-2110cc04a849 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:15.909176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb0267-cc63-4f9f-bdad-2110cc04a849 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:15.911056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 364/720] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:15.918294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121677d-6345-4eb6-8a05-dbde8cd946bf None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:15.918589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121677d-6345-4eb6-8a05-dbde8cd946bf None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:15.918871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121677d-6345-4eb6-8a05-dbde8cd946bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:15.935918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9121677d-6345-4eb6-8a05-dbde8cd946bf None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:15.937155 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9121677d-6345-4eb6-8a05-dbde8cd946bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:15.970671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:16.039297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 357/721] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:16.062240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb0267-cc63-4f9f-bdad-2110cc04a849 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:16.076509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:16.181435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb0267-cc63-4f9f-bdad-2110cc04a849 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:16.231039 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79892a89-e25a-4788-a73a-f487a66fa452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:16.231853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79892a89-e25a-4788-a73a-f487a66fa452 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:16.232141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79892a89-e25a-4788-a73a-f487a66fa452 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:16.232479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79892a89-e25a-4788-a73a-f487a66fa452 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:16.248139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79892a89-e25a-4788-a73a-f487a66fa452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-79892a89-e25a-4788-a73a-f487a66fa452 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:16.256102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:16.257199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:16.258182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:16.262033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 365/722] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:16.263655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121677d-6345-4eb6-8a05-dbde8cd946bf None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:16.390672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121677d-6345-4eb6-8a05-dbde8cd946bf None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:16.426572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341537a-4879-476f-9547-3429a621a600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:16.427628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341537a-4879-476f-9547-3429a621a600 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:16.428110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341537a-4879-476f-9547-3429a621a600 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:16.428635 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341537a-4879-476f-9547-3429a621a600 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:16.429603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341537a-4879-476f-9547-3429a621a600 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:16.430108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341537a-4879-476f-9547-3429a621a600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:16.431743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341537a-4879-476f-9547-3429a621a600 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:16.441641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 358/723] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:16.447806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79892a89-e25a-4788-a73a-f487a66fa452 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:16.560619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:16.604696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79892a89-e25a-4788-a73a-f487a66fa452 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:16.649245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597664e-7cfa-4dc4-972b-2df536477eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:16.650053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597664e-7cfa-4dc4-972b-2df536477eac None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:16.650339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597664e-7cfa-4dc4-972b-2df536477eac None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:16.650621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597664e-7cfa-4dc4-972b-2df536477eac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:16.667008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597664e-7cfa-4dc4-972b-2df536477eac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:16.668586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597664e-7cfa-4dc4-972b-2df536477eac None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c597664e-7cfa-4dc4-972b-2df536477eac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:16.687878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:16.689042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:16.690234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:16.690234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:16.690234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:16.690234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:16.696541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 366/724] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:16.701599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341537a-4879-476f-9547-3429a621a600 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:16.740768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:16.908832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c7b4d-d601-4217-9a7a-b5576cbafe78 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:16.909821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c7b4d-d601-4217-9a7a-b5576cbafe78 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:16.910303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c7b4d-d601-4217-9a7a-b5576cbafe78 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:16.916833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 367/725] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:16.925498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae4f7b-5225-4459-8991-3d75603d8fcf None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:16.926553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae4f7b-5225-4459-8991-3d75603d8fcf None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:16.927040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae4f7b-5225-4459-8991-3d75603d8fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:16.947670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341537a-4879-476f-9547-3429a621a600 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:16.952164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e1ae4f7b-5225-4459-8991-3d75603d8fcf None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:16.955713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e1ae4f7b-5225-4459-8991-3d75603d8fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:17.031593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1cf6d-a477-4880-8951-3ad52c78cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:17.037055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1cf6d-a477-4880-8951-3ad52c78cd16 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:17.039855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1cf6d-a477-4880-8951-3ad52c78cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:17.040831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1cf6d-a477-4880-8951-3ad52c78cd16 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:17.041777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1cf6d-a477-4880-8951-3ad52c78cd16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:17.042306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1cf6d-a477-4880-8951-3ad52c78cd16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:17.045127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1cf6d-a477-4880-8951-3ad52c78cd16 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:17.071163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 359/726] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:17.079072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1cf6d-a477-4880-8951-3ad52c78cd16 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:17.104477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 368/727] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 08 01:52:17.106250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a856fe8-12d4-499d-8c4d-e61fa431ac20 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:17.106744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a856fe8-12d4-499d-8c4d-e61fa431ac20 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:17.107861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a856fe8-12d4-499d-8c4d-e61fa431ac20 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:17.112231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 369/728] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:17.116206 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748a13d-298a-47d4-bfc8-1d6df212c9b5 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:17.124416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748a13d-298a-47d4-bfc8-1d6df212c9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:17.125009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748a13d-298a-47d4-bfc8-1d6df212c9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:17.151954 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-2748a13d-298a-47d4-bfc8-1d6df212c9b5 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:17.153290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2748a13d-298a-47d4-bfc8-1d6df212c9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:17.205924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:17.231194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1cf6d-a477-4880-8951-3ad52c78cd16 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:17.246473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:17.272974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85167fa-e9d3-4085-a1a3-7bcd61c412bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:17.274139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85167fa-e9d3-4085-a1a3-7bcd61c412bb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:17.274742 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85167fa-e9d3-4085-a1a3-7bcd61c412bb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:17.275214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85167fa-e9d3-4085-a1a3-7bcd61c412bb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:17.283204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85167fa-e9d3-4085-a1a3-7bcd61c412bb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:17.284957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85167fa-e9d3-4085-a1a3-7bcd61c412bb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b85167fa-e9d3-4085-a1a3-7bcd61c412bb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:17.301307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:17.302290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:17.303245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:17.303245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:17.303245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:17.303245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:17.303245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:17.303245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:17.303245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:17.303245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:17.303245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:52:17.303245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:17.304685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 370/729] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:17.305536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748a13d-298a-47d4-bfc8-1d6df212c9b5 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:17.306489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 360/730] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:17.317750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85167fa-e9d3-4085-a1a3-7bcd61c412bb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:17.445993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85167fa-e9d3-4085-a1a3-7bcd61c412bb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:17.457403 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748a13d-298a-47d4-bfc8-1d6df212c9b5 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:17.484360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fba636-adfb-4815-99d3-c021b2496f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:17.485445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fba636-adfb-4815-99d3-c021b2496f3a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:17.485953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fba636-adfb-4815-99d3-c021b2496f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:17.486449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fba636-adfb-4815-99d3-c021b2496f3a None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:17.487607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fba636-adfb-4815-99d3-c021b2496f3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:17.488164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fba636-adfb-4815-99d3-c021b2496f3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:17.489986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fba636-adfb-4815-99d3-c021b2496f3a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:17.493179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347b1f4-5562-483d-8b6d-e91b0fc08000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:17.494394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347b1f4-5562-483d-8b6d-e91b0fc08000 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:17.494394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347b1f4-5562-483d-8b6d-e91b0fc08000 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:17.494394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347b1f4-5562-483d-8b6d-e91b0fc08000 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:17.495115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347b1f4-5562-483d-8b6d-e91b0fc08000 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:17.496524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347b1f4-5562-483d-8b6d-e91b0fc08000 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:17.500321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:17.513037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 361/731] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jun 8 01:52:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:17.514223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fba636-adfb-4815-99d3-c021b2496f3a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:17.514332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 371/732] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:52:17 2022] POST /identity/v3/services => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:17.521669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347b1f4-5562-483d-8b6d-e91b0fc08000 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:17.608855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:17.636387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fba636-adfb-4815-99d3-c021b2496f3a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:17.685636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997f4e2-d274-464b-8d6f-fefbda12f537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:17.686862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997f4e2-d274-464b-8d6f-fefbda12f537 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:17.687351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997f4e2-d274-464b-8d6f-fefbda12f537 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:17.687940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997f4e2-d274-464b-8d6f-fefbda12f537 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:17.715993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997f4e2-d274-464b-8d6f-fefbda12f537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:17.730279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347b1f4-5562-483d-8b6d-e91b0fc08000 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2997f4e2-d274-464b-8d6f-fefbda12f537 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:17.734153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:17.735182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:17.736021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 362/733] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:17.740583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997f4e2-d274-464b-8d6f-fefbda12f537 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:17.776822 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec382f8-201c-4fac-8a5d-39ceba1e8f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:17.778553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec382f8-201c-4fac-8a5d-39ceba1e8f68 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:17.779088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec382f8-201c-4fac-8a5d-39ceba1e8f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:17.779571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec382f8-201c-4fac-8a5d-39ceba1e8f68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:17.858264 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:17.945214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec382f8-201c-4fac-8a5d-39ceba1e8f68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:17.946581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997f4e2-d274-464b-8d6f-fefbda12f537 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:17.947740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec382f8-201c-4fac-8a5d-39ceba1e8f68 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:17.994568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b8a1b-95eb-4268-a287-19fefc4db7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:17.995288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b8a1b-95eb-4268-a287-19fefc4db7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:17.995644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b8a1b-95eb-4268-a287-19fefc4db7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:17.995991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b8a1b-95eb-4268-a287-19fefc4db7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:17.997205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b8a1b-95eb-4268-a287-19fefc4db7d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:17.997586 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b8a1b-95eb-4268-a287-19fefc4db7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:17.998552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:17.999312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b8a1b-95eb-4268-a287-19fefc4db7d9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:18.011221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 363/734] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:18.017239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b8a1b-95eb-4268-a287-19fefc4db7d9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:18.066946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 372/735] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:17 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 08 01:52:18.073171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec382f8-201c-4fac-8a5d-39ceba1e8f68 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:18.168886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b8a1b-95eb-4268-a287-19fefc4db7d9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:18.184375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:18.210363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd70d0-81f6-4184-9521-1769173e41c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:18.212200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd70d0-81f6-4184-9521-1769173e41c3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.212754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd70d0-81f6-4184-9521-1769173e41c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.213231 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd70d0-81f6-4184-9521-1769173e41c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.218550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec382f8-201c-4fac-8a5d-39ceba1e8f68 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:18.225401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd70d0-81f6-4184-9521-1769173e41c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:18.227694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd70d0-81f6-4184-9521-1769173e41c3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b2fd70d0-81f6-4184-9521-1769173e41c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:18.246256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:18.247197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:18.248219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:18.248219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:18.248219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:18.248219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:18.250637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 364/736] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:18.259020 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd70d0-81f6-4184-9521-1769173e41c3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:18.307734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c91817-2e26-49dd-a9fc-436a9b471216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:18.308844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c91817-2e26-49dd-a9fc-436a9b471216 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.308957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c91817-2e26-49dd-a9fc-436a9b471216 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.309223 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c91817-2e26-49dd-a9fc-436a9b471216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.321543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c91817-2e26-49dd-a9fc-436a9b471216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:18.323200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c91817-2e26-49dd-a9fc-436a9b471216 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:18.324980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-f9c91817-2e26-49dd-a9fc-436a9b471216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:18.345318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:18.346284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:18.347178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:18.347178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:18.347178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:18.347178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:18.349630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 373/737] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:18.352156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738289d-a948-431a-8805-193945587bea None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.352735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738289d-a948-431a-8805-193945587bea None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.353188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738289d-a948-431a-8805-193945587bea None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.354924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 374/738] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:18.359873 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738289d-a948-431a-8805-193945587bea None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:18.373279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd70d0-81f6-4184-9521-1769173e41c3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:18.409546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3bc26-1ff1-4ca9-8e54-c676708d41ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:18.410317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3bc26-1ff1-4ca9-8e54-c676708d41ea None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.410922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3bc26-1ff1-4ca9-8e54-c676708d41ea None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.411269 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3bc26-1ff1-4ca9-8e54-c676708d41ea None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.412113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3bc26-1ff1-4ca9-8e54-c676708d41ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:18.412530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3bc26-1ff1-4ca9-8e54-c676708d41ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:18.414114 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3bc26-1ff1-4ca9-8e54-c676708d41ea None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:18.427159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 365/739] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:18.429286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4bfb6-fe3c-4e38-aaf2-f180807945a0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.429916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4bfb6-fe3c-4e38-aaf2-f180807945a0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.430013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4bfb6-fe3c-4e38-aaf2-f180807945a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.447023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-edd4bfb6-fe3c-4e38-aaf2-f180807945a0 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:18.448258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-edd4bfb6-fe3c-4e38-aaf2-f180807945a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:18.470999 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:18.483704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738289d-a948-431a-8805-193945587bea None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:18.520965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 366/740] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 08 01:52:18.522117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4bfb6-fe3c-4e38-aaf2-f180807945a0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:18.527278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5aefd-5ad4-48ff-b832-fa2e6e6badcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:18.528220 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5aefd-5ad4-48ff-b832-fa2e6e6badcb None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.528755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5aefd-5ad4-48ff-b832-fa2e6e6badcb None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.529253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5aefd-5ad4-48ff-b832-fa2e6e6badcb None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.530189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5aefd-5ad4-48ff-b832-fa2e6e6badcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:18.531207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5aefd-5ad4-48ff-b832-fa2e6e6badcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:18.532839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5aefd-5ad4-48ff-b832-fa2e6e6badcb None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:18.548586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 375/741] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:18.550234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94e6e8-ff79-4f7c-b573-c1b5c17ac30e None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.550532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94e6e8-ff79-4f7c-b573-c1b5c17ac30e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.550811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94e6e8-ff79-4f7c-b573-c1b5c17ac30e None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.552157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 376/742] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:18.554005 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94e6e8-ff79-4f7c-b573-c1b5c17ac30e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:18.617472 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:18.638421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4bfb6-fe3c-4e38-aaf2-f180807945a0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:18.666316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94e6e8-ff79-4f7c-b573-c1b5c17ac30e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:18.677564 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abed0f7-a8e1-4861-bdff-34c46e2f178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:18.679919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abed0f7-a8e1-4861-bdff-34c46e2f178e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.680416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abed0f7-a8e1-4861-bdff-34c46e2f178e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.680836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abed0f7-a8e1-4861-bdff-34c46e2f178e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.688408 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abed0f7-a8e1-4861-bdff-34c46e2f178e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:18.690144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abed0f7-a8e1-4861-bdff-34c46e2f178e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:18.701063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7a422-4ced-4a21-8905-87c399b5385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:18.701734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7a422-4ced-4a21-8905-87c399b5385d None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.702023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7a422-4ced-4a21-8905-87c399b5385d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.702306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7a422-4ced-4a21-8905-87c399b5385d None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.703030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7a422-4ced-4a21-8905-87c399b5385d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:18.704436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7a422-4ced-4a21-8905-87c399b5385d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5abed0f7-a8e1-4861-bdff-34c46e2f178e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:18.706644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:18.707531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:18.708497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 367/743] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jun 8 01:52:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:18.709089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abed0f7-a8e1-4861-bdff-34c46e2f178e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:18.732454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 377/744] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:52:18 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:18.732454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058ae35-3157-4a09-a619-3ef837668dd5 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.732454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058ae35-3157-4a09-a619-3ef837668dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.732454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058ae35-3157-4a09-a619-3ef837668dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.738732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:18.750590 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3058ae35-3157-4a09-a619-3ef837668dd5 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:18.751833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3058ae35-3157-4a09-a619-3ef837668dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:18.854238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abed0f7-a8e1-4861-bdff-34c46e2f178e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:18.860716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 378/745] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:18.861955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058ae35-3157-4a09-a619-3ef837668dd5 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:18.893900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81c9a9-43ff-409c-ac1e-ba84a4d88727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:18.894956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81c9a9-43ff-409c-ac1e-ba84a4d88727 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:18.895344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81c9a9-43ff-409c-ac1e-ba84a4d88727 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:18.895746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81c9a9-43ff-409c-ac1e-ba84a4d88727 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:18.903827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81c9a9-43ff-409c-ac1e-ba84a4d88727 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:18.905873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81c9a9-43ff-409c-ac1e-ba84a4d88727 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:18.908266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0d81c9a9-43ff-409c-ac1e-ba84a4d88727 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:18.925116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:18.926143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:18.927034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:18.927034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:18.927034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:18.927034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:18.927034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:18.927034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:18.927034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:18.927034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:18.927034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 368/746] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:18.932946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81c9a9-43ff-409c-ac1e-ba84a4d88727 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:18.979374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058ae35-3157-4a09-a619-3ef837668dd5 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:19.023751 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18421df-4b0c-495b-bb3a-abde8d75c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:19.025259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18421df-4b0c-495b-bb3a-abde8d75c9cd None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:19.025259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18421df-4b0c-495b-bb3a-abde8d75c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:19.025259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18421df-4b0c-495b-bb3a-abde8d75c9cd None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:19.025966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18421df-4b0c-495b-bb3a-abde8d75c9cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:19.026175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18421df-4b0c-495b-bb3a-abde8d75c9cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:19.027644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18421df-4b0c-495b-bb3a-abde8d75c9cd None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:19.028828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:19.048005 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 379/747] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jun 8 01:52:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:19.048903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18421df-4b0c-495b-bb3a-abde8d75c9cd None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:19.070865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81c9a9-43ff-409c-ac1e-ba84a4d88727 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:19.111573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550629ef-e73c-467f-9bbd-a189d5c2a994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:19.112300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550629ef-e73c-467f-9bbd-a189d5c2a994 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:19.112613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550629ef-e73c-467f-9bbd-a189d5c2a994 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:19.112888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550629ef-e73c-467f-9bbd-a189d5c2a994 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:19.124276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550629ef-e73c-467f-9bbd-a189d5c2a994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-550629ef-e73c-467f-9bbd-a189d5c2a994 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:19.132044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:19.133026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:19.133865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 369/748] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:19.134577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550629ef-e73c-467f-9bbd-a189d5c2a994 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:19.164663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:19.170775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18421df-4b0c-495b-bb3a-abde8d75c9cd None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:19.212161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1503153-7943-4a89-938d-6444adcd52f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:19.213287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1503153-7943-4a89-938d-6444adcd52f7 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:19.213771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1503153-7943-4a89-938d-6444adcd52f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:19.214269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1503153-7943-4a89-938d-6444adcd52f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:19.215944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1503153-7943-4a89-938d-6444adcd52f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:19.216489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1503153-7943-4a89-938d-6444adcd52f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:19.219711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1503153-7943-4a89-938d-6444adcd52f7 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:19.243785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 380/749] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:19.243785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1503153-7943-4a89-938d-6444adcd52f7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:19.259667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550629ef-e73c-467f-9bbd-a189d5c2a994 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:19.305583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfae5a-125a-44e4-9b11-2846bc9d55ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:19.307003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfae5a-125a-44e4-9b11-2846bc9d55ff None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:19.307899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfae5a-125a-44e4-9b11-2846bc9d55ff None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:19.308663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfae5a-125a-44e4-9b11-2846bc9d55ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:19.312448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:19.325053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfae5a-125a-44e4-9b11-2846bc9d55ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:19.326802 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfae5a-125a-44e4-9b11-2846bc9d55ff None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-fcdfae5a-125a-44e4-9b11-2846bc9d55ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:19.349902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:19.351185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:19.352077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:19.352077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:19.352077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:19.352077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:19.354499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 370/750] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:19.355433 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfae5a-125a-44e4-9b11-2846bc9d55ff None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:19.363712 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1503153-7943-4a89-938d-6444adcd52f7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:19.402938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6981d-8294-4b45-aa05-64a612dce3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:19.404349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6981d-8294-4b45-aa05-64a612dce3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:19.404812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6981d-8294-4b45-aa05-64a612dce3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:19.405257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6981d-8294-4b45-aa05-64a612dce3b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:19.406595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6981d-8294-4b45-aa05-64a612dce3b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:19.407073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6981d-8294-4b45-aa05-64a612dce3b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:19.409282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6981d-8294-4b45-aa05-64a612dce3b5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:19.421256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 381/751] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:52:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:19.422274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6981d-8294-4b45-aa05-64a612dce3b5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:19.451059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:19.477498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfae5a-125a-44e4-9b11-2846bc9d55ff None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:19.520375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07d70d-a360-4eea-af99-737efdfab209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:19.521420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07d70d-a360-4eea-af99-737efdfab209 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:19.521944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07d70d-a360-4eea-af99-737efdfab209 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:19.522990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07d70d-a360-4eea-af99-737efdfab209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:19.524209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07d70d-a360-4eea-af99-737efdfab209 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=fc299258fe814636880cf595fc2904b4) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:19.524746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07d70d-a360-4eea-af99-737efdfab209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:19.526408 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07d70d-a360-4eea-af99-737efdfab209 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:19.532771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 371/752] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:19 2022] GET /identity/v3/role_assignments?user.id=fc299258fe814636880cf595fc2904b4&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:19.533919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07d70d-a360-4eea-af99-737efdfab209 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:19.580338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6981d-8294-4b45-aa05-64a612dce3b5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:19.625923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f49983-a9e4-4987-84a6-7d4a0e9642c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:19.627109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f49983-a9e4-4987-84a6-7d4a0e9642c5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:19.627619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f49983-a9e4-4987-84a6-7d4a0e9642c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:19.628103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f49983-a9e4-4987-84a6-7d4a0e9642c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:19.629108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f49983-a9e4-4987-84a6-7d4a0e9642c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:19.629652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f49983-a9e4-4987-84a6-7d4a0e9642c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:19.631317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f49983-a9e4-4987-84a6-7d4a0e9642c5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:19.659694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 382/753] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:19.661281 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f49983-a9e4-4987-84a6-7d4a0e9642c5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:19.661774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:19.763119 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07d70d-a360-4eea-af99-737efdfab209 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:19.824565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:19.829381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cf988-2d97-4684-b75a-f57afeb35181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:19.830467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cf988-2d97-4684-b75a-f57afeb35181 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:19.830957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cf988-2d97-4684-b75a-f57afeb35181 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:19.831435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9cf988-2d97-4684-b75a-f57afeb35181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:19.846626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cf988-2d97-4684-b75a-f57afeb35181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:19.848466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9cf988-2d97-4684-b75a-f57afeb35181 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:19.853128 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f49983-a9e4-4987-84a6-7d4a0e9642c5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-dd9cf988-2d97-4684-b75a-f57afeb35181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:19.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:19.869702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:19.870593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:19.870593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:19.870593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:19.870593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:19.878687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 372/754] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:19.880192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cf988-2d97-4684-b75a-f57afeb35181 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:19.908447 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a984c04-bb8b-4abd-825b-8466a2454c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:19.909529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a984c04-bb8b-4abd-825b-8466a2454c43 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:19.909964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a984c04-bb8b-4abd-825b-8466a2454c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:19.910389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a984c04-bb8b-4abd-825b-8466a2454c43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:19.925749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a984c04-bb8b-4abd-825b-8466a2454c43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:19.927247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a984c04-bb8b-4abd-825b-8466a2454c43 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-5a984c04-bb8b-4abd-825b-8466a2454c43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:19.948362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:19.949467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:19.950360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:19.950360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:19.950360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:19.950360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:19.950360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:19.950360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:19.950360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:19.950360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:19.961552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 383/755] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:19.969907 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a984c04-bb8b-4abd-825b-8466a2454c43 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:20.077902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9cf988-2d97-4684-b75a-f57afeb35181 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:20.090583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:20.139943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a984c04-bb8b-4abd-825b-8466a2454c43 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:20.169118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaccc4-b095-4cf3-87d8-46baca7d8800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:20.170123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaccc4-b095-4cf3-87d8-46baca7d8800 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:20.170688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaccc4-b095-4cf3-87d8-46baca7d8800 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:20.171149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaccc4-b095-4cf3-87d8-46baca7d8800 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:20.172059 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaccc4-b095-4cf3-87d8-46baca7d8800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:20.172575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaccc4-b095-4cf3-87d8-46baca7d8800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:20.174154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaccc4-b095-4cf3-87d8-46baca7d8800 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:20.180422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fb3c0-e2e6-4309-a02e-aab6311edb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:20.180422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fb3c0-e2e6-4309-a02e-aab6311edb30 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:20.180422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fb3c0-e2e6-4309-a02e-aab6311edb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:20.180422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fb3c0-e2e6-4309-a02e-aab6311edb30 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:20.180794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fb3c0-e2e6-4309-a02e-aab6311edb30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:20.180794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fb3c0-e2e6-4309-a02e-aab6311edb30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:20.181980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fb3c0-e2e6-4309-a02e-aab6311edb30 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:20.203868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 373/756] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:20.206555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:20.215704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 384/757] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:20.217120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaccc4-b095-4cf3-87d8-46baca7d8800 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:20.217259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fb3c0-e2e6-4309-a02e-aab6311edb30 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:20.323407 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fb3c0-e2e6-4309-a02e-aab6311edb30 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:20.360209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91a850-1765-4d87-8006-22b777410d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:20.361233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91a850-1765-4d87-8006-22b777410d9c None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:20.361567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91a850-1765-4d87-8006-22b777410d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:20.361845 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91a850-1765-4d87-8006-22b777410d9c None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/0ef930dc7b7a4fb29f7b32e464e9eb3b/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:20.363886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91a850-1765-4d87-8006-22b777410d9c None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=0ef930dc7b7a4fb29f7b32e464e9eb3b, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:20.365347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91a850-1765-4d87-8006-22b777410d9c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:20.387374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaccc4-b095-4cf3-87d8-46baca7d8800 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:20.392668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 385/758] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:52:20 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/0ef930dc7b7a4fb29f7b32e464e9eb3b/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:52:20.395549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305d719-e459-4d4a-bc56-2f4738d0184a None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:20.399190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305d719-e459-4d4a-bc56-2f4738d0184a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:20.399779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305d719-e459-4d4a-bc56-2f4738d0184a None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:20.403096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 386/759] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:20.407290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5ffa-558c-43fb-a0b2-da079973b82b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:20.407913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5ffa-558c-43fb-a0b2-da079973b82b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:20.408387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5ffa-558c-43fb-a0b2-da079973b82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:20.441127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-194f5ffa-558c-43fb-a0b2-da079973b82b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:20.443956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-194f5ffa-558c-43fb-a0b2-da079973b82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:20.472968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13731c5-6f13-4673-bf77-daf87ab964fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:20.474241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13731c5-6f13-4673-bf77-daf87ab964fb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:20.474751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13731c5-6f13-4673-bf77-daf87ab964fb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:20.475372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13731c5-6f13-4673-bf77-daf87ab964fb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:20.495182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13731c5-6f13-4673-bf77-daf87ab964fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:20.497112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13731c5-6f13-4673-bf77-daf87ab964fb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-c13731c5-6f13-4673-bf77-daf87ab964fb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:20.530275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:20.531293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:20.532523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:20.532523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:20.532523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:20.532523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:20.532523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:20.532523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:20.532523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:20.532523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:20.532523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:20.532523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:20.535154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 374/760] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jun 8 01:52:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:20.547072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:20.551627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13731c5-6f13-4673-bf77-daf87ab964fb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:20.564527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 387/761] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:20.587214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5ffa-558c-43fb-a0b2-da079973b82b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:20.609857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:20.734121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5ffa-558c-43fb-a0b2-da079973b82b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:20.736156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13731c5-6f13-4673-bf77-daf87ab964fb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:20.771771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9161156-873a-418d-b486-2d3290208d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:20.773320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9161156-873a-418d-b486-2d3290208d69 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:20.773694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9161156-873a-418d-b486-2d3290208d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:20.773995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9161156-873a-418d-b486-2d3290208d69 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:20.774805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9161156-873a-418d-b486-2d3290208d69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:20.775111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9161156-873a-418d-b486-2d3290208d69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:20.776291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad39d3-d876-46ae-a20a-79db67a006d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:20.776598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9161156-873a-418d-b486-2d3290208d69 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:20.777316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad39d3-d876-46ae-a20a-79db67a006d9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:20.777699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad39d3-d876-46ae-a20a-79db67a006d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:20.778167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad39d3-d876-46ae-a20a-79db67a006d9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:20.778927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad39d3-d876-46ae-a20a-79db67a006d9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:20.780486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad39d3-d876-46ae-a20a-79db67a006d9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-19ad39d3-d876-46ae-a20a-79db67a006d9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:20.795201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:20.796381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 08 01:52:20.797703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:20.798687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 375/762] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jun 8 01:52:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:20.798774 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 388/763] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Jun 8 01:52:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:20.802600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9161156-873a-418d-b486-2d3290208d69 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:20.805047 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad39d3-d876-46ae-a20a-79db67a006d9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:20.902817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:20.930385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9161156-873a-418d-b486-2d3290208d69 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:20.952746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:20.996924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30815f56-6597-4ca7-8e0e-d602343610c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:20.997617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30815f56-6597-4ca7-8e0e-d602343610c5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:20.997901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30815f56-6597-4ca7-8e0e-d602343610c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:20.998183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30815f56-6597-4ca7-8e0e-d602343610c5 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:20.998897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30815f56-6597-4ca7-8e0e-d602343610c5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:20.999204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30815f56-6597-4ca7-8e0e-d602343610c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:21.000611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30815f56-6597-4ca7-8e0e-d602343610c5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:21.012852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 376/764] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jun 8 01:52:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:21.014351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30815f56-6597-4ca7-8e0e-d602343610c5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:21.058669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad39d3-d876-46ae-a20a-79db67a006d9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:21.124887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af905a10-00df-4ff8-b7c1-8f796702f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:21.125545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af905a10-00df-4ff8-b7c1-8f796702f6af None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:21.125826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af905a10-00df-4ff8-b7c1-8f796702f6af None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:21.126104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af905a10-00df-4ff8-b7c1-8f796702f6af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:21.190350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30815f56-6597-4ca7-8e0e-d602343610c5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:21.222260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af905a10-00df-4ff8-b7c1-8f796702f6af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:21.223748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af905a10-00df-4ff8-b7c1-8f796702f6af None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:21.283856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eb09c-afeb-4dbf-9f2d-e58ba74e31b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:21.284482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eb09c-afeb-4dbf-9f2d-e58ba74e31b0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:21.284769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eb09c-afeb-4dbf-9f2d-e58ba74e31b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:21.285047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eb09c-afeb-4dbf-9f2d-e58ba74e31b0 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:21.286017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eb09c-afeb-4dbf-9f2d-e58ba74e31b0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:21.286327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eb09c-afeb-4dbf-9f2d-e58ba74e31b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:21.286432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:21.287777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eb09c-afeb-4dbf-9f2d-e58ba74e31b0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:21.302444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 377/765] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jun 8 01:52:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:21.310532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eb09c-afeb-4dbf-9f2d-e58ba74e31b0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:21.325175 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:21.373343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 389/766] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:52:21.378577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af905a10-00df-4ff8-b7c1-8f796702f6af None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:21.430716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eb09c-afeb-4dbf-9f2d-e58ba74e31b0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:21.478164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7ddfc-3519-40be-9664-1bc6480cb594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:21.479677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7ddfc-3519-40be-9664-1bc6480cb594 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:21.480166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7ddfc-3519-40be-9664-1bc6480cb594 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:21.480672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7ddfc-3519-40be-9664-1bc6480cb594 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:21.482325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7ddfc-3519-40be-9664-1bc6480cb594 None admin] RBAC: Adding query filter params (interface=public, service_id=1db86b8682c54687aec5035e57545cc3, region_id=RegionOne) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:21.482869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7ddfc-3519-40be-9664-1bc6480cb594 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:21.485233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7ddfc-3519-40be-9664-1bc6480cb594 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:21.496937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 378/767] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jun 8 01:52:21 2022] GET /identity/v3/endpoints?service_id=1db86b8682c54687aec5035e57545cc3&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:21.498549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cf90f-cc92-4550-ac45-c73941f76e86 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:21.499038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cf90f-cc92-4550-ac45-c73941f76e86 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:21.499508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cf90f-cc92-4550-ac45-c73941f76e86 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:21.505158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 379/768] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:21.505388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cf90f-cc92-4550-ac45-c73941f76e86 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:21.566639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af905a10-00df-4ff8-b7c1-8f796702f6af None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:21.607472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:21.616310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2049082-1541-4908-be8a-da26c01cc80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:21.617593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2049082-1541-4908-be8a-da26c01cc80a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:21.617888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2049082-1541-4908-be8a-da26c01cc80a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:21.618172 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2049082-1541-4908-be8a-da26c01cc80a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:21.633184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2049082-1541-4908-be8a-da26c01cc80a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:21.635369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2049082-1541-4908-be8a-da26c01cc80a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:21.638153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cf90f-cc92-4550-ac45-c73941f76e86 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c2049082-1541-4908-be8a-da26c01cc80a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:21.659335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:21.660291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:21.661476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:21.661476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:21.661476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:21.661476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:21.661476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 390/769] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:21.662479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbc947-feed-40fb-9ceb-582ac38345af None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:21.663319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbc947-feed-40fb-9ceb-582ac38345af None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:21.663600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbc947-feed-40fb-9ceb-582ac38345af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:21.677514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:21.684909 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-cedbc947-feed-40fb-9ceb-582ac38345af None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:21.685503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04359b7b-1e72-4bc8-ba61-81cc6f566747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:21.686180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04359b7b-1e72-4bc8-ba61-81cc6f566747 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:21.686466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04359b7b-1e72-4bc8-ba61-81cc6f566747 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:21.686744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04359b7b-1e72-4bc8-ba61-81cc6f566747 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:21.687330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-cedbc947-feed-40fb-9ceb-582ac38345af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:21.687454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04359b7b-1e72-4bc8-ba61-81cc6f566747 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:21.688844 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04359b7b-1e72-4bc8-ba61-81cc6f566747 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:21.700483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 380/770] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jun 8 01:52:21 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:21.701779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04359b7b-1e72-4bc8-ba61-81cc6f566747 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:21.837200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 391/771] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:21.888761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbc947-feed-40fb-9ceb-582ac38345af None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:21.934813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04359b7b-1e72-4bc8-ba61-81cc6f566747 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:21.993417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc313011-4f57-4937-8831-447134028acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:21.994104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc313011-4f57-4937-8831-447134028acc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:21.994392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc313011-4f57-4937-8831-447134028acc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:21.994680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc313011-4f57-4937-8831-447134028acc None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:21.995400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc313011-4f57-4937-8831-447134028acc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:21.995709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc313011-4f57-4937-8831-447134028acc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:21.997656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc313011-4f57-4937-8831-447134028acc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:22.007797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:22.020778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 381/772] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:22.021971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc313011-4f57-4937-8831-447134028acc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:22.065488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbc947-feed-40fb-9ceb-582ac38345af None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:22.111848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523f45f-d700-40d0-a2a0-33203fae886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:22.112599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523f45f-d700-40d0-a2a0-33203fae886b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.112886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523f45f-d700-40d0-a2a0-33203fae886b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.113167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523f45f-d700-40d0-a2a0-33203fae886b None admin] PATH_INFO: `/v3/services/network` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.113828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523f45f-d700-40d0-a2a0-33203fae886b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:22.115212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523f45f-d700-40d0-a2a0-33203fae886b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:22.118785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-2523f45f-d700-40d0-a2a0-33203fae886b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:22.129001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:22.130212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 08 01:52:22.131268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:22.135482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 392/773] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:21 2022] GET /identity/v3/services/network => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:22.137588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f4a49-2618-4003-ab3e-04214f3450cf None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.137874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f4a49-2618-4003-ab3e-04214f3450cf None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.138152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f4a49-2618-4003-ab3e-04214f3450cf None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.139538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 393/774] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:22.145155 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f4a49-2618-4003-ab3e-04214f3450cf None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:22.155472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc313011-4f57-4937-8831-447134028acc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:22.196133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4134ae1-2758-4694-8a9c-303c6a6e853e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:22.197329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4134ae1-2758-4694-8a9c-303c6a6e853e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.197818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4134ae1-2758-4694-8a9c-303c6a6e853e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.198294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4134ae1-2758-4694-8a9c-303c6a6e853e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4134ae1-2758-4694-8a9c-303c6a6e853e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:22.207376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4134ae1-2758-4694-8a9c-303c6a6e853e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d4134ae1-2758-4694-8a9c-303c6a6e853e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:22.223493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:22.224453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:22.226196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:22.226196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:22.226196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:22.226196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:22.226196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:22.226196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:22.226196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:22.226196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:22.233448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 382/775] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:22.235118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d58bf0-3091-4a39-9b24-e6be62a94744 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.235633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d58bf0-3091-4a39-9b24-e6be62a94744 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.236111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d58bf0-3091-4a39-9b24-e6be62a94744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.254230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c8d58bf0-3091-4a39-9b24-e6be62a94744 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:22.255756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c8d58bf0-3091-4a39-9b24-e6be62a94744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:22.293322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f4a49-2618-4003-ab3e-04214f3450cf None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:22.323319 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:22.331091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e818f-8b57-4c46-8731-644527aea553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:22.332222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e818f-8b57-4c46-8731-644527aea553 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.332743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e818f-8b57-4c46-8731-644527aea553 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.333265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e818f-8b57-4c46-8731-644527aea553 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.334175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e818f-8b57-4c46-8731-644527aea553 None admin] RBAC: Adding query filter params (name=network) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:22.335429 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e818f-8b57-4c46-8731-644527aea553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:22.337103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e818f-8b57-4c46-8731-644527aea553 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:22.348420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 383/776] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:22.348420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d58bf0-3091-4a39-9b24-e6be62a94744 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:22.351234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 394/777] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:22.356229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e818f-8b57-4c46-8731-644527aea553 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:22.442483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:22.471237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d58bf0-3091-4a39-9b24-e6be62a94744 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:22.484692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e818f-8b57-4c46-8731-644527aea553 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:22.511574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a7f86-cb27-4aab-8bbd-fd9b35258675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:22.512683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a7f86-cb27-4aab-8bbd-fd9b35258675 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.513651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a7f86-cb27-4aab-8bbd-fd9b35258675 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.514480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a7f86-cb27-4aab-8bbd-fd9b35258675 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.515501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a7f86-cb27-4aab-8bbd-fd9b35258675 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:22.516017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a7f86-cb27-4aab-8bbd-fd9b35258675 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:22.518016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a7f86-cb27-4aab-8bbd-fd9b35258675 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:22.519870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdce32-1f35-4799-9c16-148f64b9bae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:22.520582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdce32-1f35-4799-9c16-148f64b9bae5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.520920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdce32-1f35-4799-9c16-148f64b9bae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.521195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdce32-1f35-4799-9c16-148f64b9bae5 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.521901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdce32-1f35-4799-9c16-148f64b9bae5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:22.523222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdce32-1f35-4799-9c16-148f64b9bae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:22.524688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdce32-1f35-4799-9c16-148f64b9bae5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:22.538887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 395/779] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:22.538887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdce32-1f35-4799-9c16-148f64b9bae5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:22.539356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 384/779] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:22.540757 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a7f86-cb27-4aab-8bbd-fd9b35258675 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:22.597432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:22.641585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdce32-1f35-4799-9c16-148f64b9bae5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:22.658540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a7f86-cb27-4aab-8bbd-fd9b35258675 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:22.670668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:22.678964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a12017-1bfa-46ba-a993-18121a849a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:22.680381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a12017-1bfa-46ba-a993-18121a849a54 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.681236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a12017-1bfa-46ba-a993-18121a849a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.681521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a12017-1bfa-46ba-a993-18121a849a54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.692598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a12017-1bfa-46ba-a993-18121a849a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-76a12017-1bfa-46ba-a993-18121a849a54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:22.701088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:22.702065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:22.702896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:22.707107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 396/780] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:22.707820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a12017-1bfa-46ba-a993-18121a849a54 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:22.708097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5a542-9b6c-4a2a-ad7f-8b586e827d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:22.709298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5a542-9b6c-4a2a-ad7f-8b586e827d7d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.709885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5a542-9b6c-4a2a-ad7f-8b586e827d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.710424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5a542-9b6c-4a2a-ad7f-8b586e827d7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.711500 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5a542-9b6c-4a2a-ad7f-8b586e827d7d None admin] RBAC: Adding query filter params (interface=public, service_id=32ec27927ab24cdfbceb69ec24dd07dd, region_id=RegionOne) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:22.712095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5a542-9b6c-4a2a-ad7f-8b586e827d7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:22.713849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5a542-9b6c-4a2a-ad7f-8b586e827d7d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:22.720584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 385/781] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jun 8 01:52:22 2022] GET /identity/v3/endpoints?service_id=32ec27927ab24cdfbceb69ec24dd07dd&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:22.721741 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5a542-9b6c-4a2a-ad7f-8b586e827d7d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:22.821179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a12017-1bfa-46ba-a993-18121a849a54 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:22.842020 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5a542-9b6c-4a2a-ad7f-8b586e827d7d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:22.857944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3594c8-0dde-4a00-8c56-db4c4b981f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:22.858135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:22.858598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3594c8-0dde-4a00-8c56-db4c4b981f27 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.858895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3594c8-0dde-4a00-8c56-db4c4b981f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.859405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3594c8-0dde-4a00-8c56-db4c4b981f27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.859986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3594c8-0dde-4a00-8c56-db4c4b981f27 None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=a6aa350eb1504e9c9be9c90870779381) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:22.860297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3594c8-0dde-4a00-8c56-db4c4b981f27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:22.862018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3594c8-0dde-4a00-8c56-db4c4b981f27 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:22.874000 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 397/782] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:22 2022] GET /identity/v3/role_assignments?user.id=a6aa350eb1504e9c9be9c90870779381&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:22.874000 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3594c8-0dde-4a00-8c56-db4c4b981f27 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:22.904610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c13bc1-31aa-490b-a01c-a54f94fed434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:22.905733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c13bc1-31aa-490b-a01c-a54f94fed434 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:22.906284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c13bc1-31aa-490b-a01c-a54f94fed434 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:22.906794 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c13bc1-31aa-490b-a01c-a54f94fed434 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:22.908434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c13bc1-31aa-490b-a01c-a54f94fed434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:22.908953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c13bc1-31aa-490b-a01c-a54f94fed434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:22.910816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c13bc1-31aa-490b-a01c-a54f94fed434 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:22.927805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:22.929858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 386/783] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:22.938274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c13bc1-31aa-490b-a01c-a54f94fed434 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:22.988055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3594c8-0dde-4a00-8c56-db4c4b981f27 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:23.029993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cbd1f-f73a-428d-bbc2-6185af307b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:23.030686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cbd1f-f73a-428d-bbc2-6185af307b2b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:23.030969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cbd1f-f73a-428d-bbc2-6185af307b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:23.031244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cbd1f-f73a-428d-bbc2-6185af307b2b None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:23.031953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cbd1f-f73a-428d-bbc2-6185af307b2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:23.033404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cbd1f-f73a-428d-bbc2-6185af307b2b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:23.046176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 398/784] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jun 8 01:52:22 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:23.109067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c13bc1-31aa-490b-a01c-a54f94fed434 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:23.109296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e850e4-2290-4b79-8fcc-4c2e79168db5 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:23.109390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e850e4-2290-4b79-8fcc-4c2e79168db5 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:23.109683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e850e4-2290-4b79-8fcc-4c2e79168db5 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:23.115691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 399/785] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:23.117838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce2047-8a8b-4d51-8473-14ddc687a938 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:23.118137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce2047-8a8b-4d51-8473-14ddc687a938 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:23.118428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce2047-8a8b-4d51-8473-14ddc687a938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:23.155920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-48ce2047-8a8b-4d51-8473-14ddc687a938 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:23.157232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-48ce2047-8a8b-4d51-8473-14ddc687a938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:23.193723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a00b4-80ef-40f2-93ae-6597bfed921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:23.194607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a00b4-80ef-40f2-93ae-6597bfed921e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:23.194900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a00b4-80ef-40f2-93ae-6597bfed921e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:23.195181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a00b4-80ef-40f2-93ae-6597bfed921e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:23.217426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a00b4-80ef-40f2-93ae-6597bfed921e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:23.219763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a00b4-80ef-40f2-93ae-6597bfed921e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-844a00b4-80ef-40f2-93ae-6597bfed921e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:23.249554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:23.250522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:23.251440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:23.251440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:23.251440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:23.251440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:23.254584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 387/786] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:23.260684 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a00b4-80ef-40f2-93ae-6597bfed921e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:23.291200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 400/787] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:23.318069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce2047-8a8b-4d51-8473-14ddc687a938 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:23.330378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:23.370814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:23.407087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a00b4-80ef-40f2-93ae-6597bfed921e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:23.430691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce2047-8a8b-4d51-8473-14ddc687a938 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:23.445079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e060b-e045-4857-8aa5-8c364d09b69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:23.445762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e060b-e045-4857-8aa5-8c364d09b69f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:23.446053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e060b-e045-4857-8aa5-8c364d09b69f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:23.446334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e060b-e045-4857-8aa5-8c364d09b69f None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:23.447058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e060b-e045-4857-8aa5-8c364d09b69f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:23.447375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e060b-e045-4857-8aa5-8c364d09b69f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:23.448792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e060b-e045-4857-8aa5-8c364d09b69f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:23.461839 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 388/788] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:23.467005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e060b-e045-4857-8aa5-8c364d09b69f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:23.490345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b7ae3-7279-48ef-a5ff-abdba87dbee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:23.491695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7ae3-7279-48ef-a5ff-abdba87dbee2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:23.491985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7ae3-7279-48ef-a5ff-abdba87dbee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:23.492476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7ae3-7279-48ef-a5ff-abdba87dbee2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:23.507497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b7ae3-7279-48ef-a5ff-abdba87dbee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-cc9b7ae3-7279-48ef-a5ff-abdba87dbee2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:23.517970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:23.518978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:23.519823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:23.524384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 401/789] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:23.525758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b7ae3-7279-48ef-a5ff-abdba87dbee2 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:23.626599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:23.650068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e060b-e045-4857-8aa5-8c364d09b69f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:23.686753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd24232-206a-4ccb-b739-463b0e66300c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:23.687621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd24232-206a-4ccb-b739-463b0e66300c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:23.687918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd24232-206a-4ccb-b739-463b0e66300c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:23.688202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd24232-206a-4ccb-b739-463b0e66300c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:23.696008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd24232-206a-4ccb-b739-463b0e66300c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:23.697673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd24232-206a-4ccb-b739-463b0e66300c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-8cd24232-206a-4ccb-b739-463b0e66300c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:23.714189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:23.715261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:23.716202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 389/790] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:23.722888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd24232-206a-4ccb-b739-463b0e66300c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:23.747868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b7ae3-7279-48ef-a5ff-abdba87dbee2 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:23.761499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:23.844893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd24232-206a-4ccb-b739-463b0e66300c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:23.853744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a191bb-86a8-4431-a21d-9ca9881f4756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:23.854797 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a191bb-86a8-4431-a21d-9ca9881f4756 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:23.854797 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a191bb-86a8-4431-a21d-9ca9881f4756 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:23.855047 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a191bb-86a8-4431-a21d-9ca9881f4756 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:23.855811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a191bb-86a8-4431-a21d-9ca9881f4756 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:23.856128 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a191bb-86a8-4431-a21d-9ca9881f4756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:23.861693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a191bb-86a8-4431-a21d-9ca9881f4756 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:23.879123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 402/791] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:23.884749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d7ce0-e675-4192-b056-f5b817804ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:23.885381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a191bb-86a8-4431-a21d-9ca9881f4756 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:23.885494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d7ce0-e675-4192-b056-f5b817804ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:23.885710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d7ce0-e675-4192-b056-f5b817804ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:23.885988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d7ce0-e675-4192-b056-f5b817804ba1 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:23.886833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d7ce0-e675-4192-b056-f5b817804ba1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:23.887142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d7ce0-e675-4192-b056-f5b817804ba1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:23.888617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d7ce0-e675-4192-b056-f5b817804ba1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:23.907496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 390/792] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jun 8 01:52:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:23.914729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d7ce0-e675-4192-b056-f5b817804ba1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:23.928751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:24.075523 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a191bb-86a8-4431-a21d-9ca9881f4756 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:24.116901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba1fe5-d73c-4021-a2e0-cdae01121876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:24.117716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba1fe5-d73c-4021-a2e0-cdae01121876 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.118000 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba1fe5-d73c-4021-a2e0-cdae01121876 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.118275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba1fe5-d73c-4021-a2e0-cdae01121876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.134377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d7ce0-e675-4192-b056-f5b817804ba1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:24.146641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba1fe5-d73c-4021-a2e0-cdae01121876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:24.148317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba1fe5-d73c-4021-a2e0-cdae01121876 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-5cba1fe5-d73c-4021-a2e0-cdae01121876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:24.176348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:24.177317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:24.178308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:24.178308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:24.178308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:24.178308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:24.188827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 403/793] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:24.189733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba1fe5-d73c-4021-a2e0-cdae01121876 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:24.207074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b46d1-7cbd-479b-a32e-f2cb6c1edd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:24.207884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b46d1-7cbd-479b-a32e-f2cb6c1edd76 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.208165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b46d1-7cbd-479b-a32e-f2cb6c1edd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.208457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b46d1-7cbd-479b-a32e-f2cb6c1edd76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.233198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b46d1-7cbd-479b-a32e-f2cb6c1edd76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:24.233379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:24.235097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b46d1-7cbd-479b-a32e-f2cb6c1edd76 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a79b46d1-7cbd-479b-a32e-f2cb6c1edd76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:24.269969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:24.270920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:24.271798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:24.271798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:24.271798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:24.271798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:24.274955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 391/794] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:24.280456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b46d1-7cbd-479b-a32e-f2cb6c1edd76 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:24.321345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba1fe5-d73c-4021-a2e0-cdae01121876 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:24.361870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97aa357-2c5c-4247-a48f-497a8a463c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:24.362475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97aa357-2c5c-4247-a48f-497a8a463c3e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.362762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97aa357-2c5c-4247-a48f-497a8a463c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.363043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97aa357-2c5c-4247-a48f-497a8a463c3e None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.368421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97aa357-2c5c-4247-a48f-497a8a463c3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:24.368421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97aa357-2c5c-4247-a48f-497a8a463c3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:24.368421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97aa357-2c5c-4247-a48f-497a8a463c3e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:24.380903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 404/795] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:24.382946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cd3a1-c626-43d6-a58b-f5445644d073 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.383555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cd3a1-c626-43d6-a58b-f5445644d073 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.383842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cd3a1-c626-43d6-a58b-f5445644d073 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.385617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 405/796] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:24.386326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cd3a1-c626-43d6-a58b-f5445644d073 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:24.401768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:24.452206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b46d1-7cbd-479b-a32e-f2cb6c1edd76 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:24.485601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:24.503127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36bd41-bc35-4425-b96f-7bf6fc66f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:24.504437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36bd41-bc35-4425-b96f-7bf6fc66f22c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.505374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36bd41-bc35-4425-b96f-7bf6fc66f22c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.505824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36bd41-bc35-4425-b96f-7bf6fc66f22c None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.507161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36bd41-bc35-4425-b96f-7bf6fc66f22c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:24.507637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36bd41-bc35-4425-b96f-7bf6fc66f22c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:24.510143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36bd41-bc35-4425-b96f-7bf6fc66f22c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:24.512754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cd3a1-c626-43d6-a58b-f5445644d073 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:24.526622 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 392/797] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:24.528126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c5095-28b9-476c-826e-655ffdc0121e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.528606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c5095-28b9-476c-826e-655ffdc0121e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.529034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c5095-28b9-476c-826e-655ffdc0121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.551253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-227c5095-28b9-476c-826e-655ffdc0121e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:24.552683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-227c5095-28b9-476c-826e-655ffdc0121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:24.564501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38deda-6289-468c-ac45-06eb7627c85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:24.565881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38deda-6289-468c-ac45-06eb7627c85d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.566552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38deda-6289-468c-ac45-06eb7627c85d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.566939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38deda-6289-468c-ac45-06eb7627c85d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.578036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38deda-6289-468c-ac45-06eb7627c85d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:24.580353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38deda-6289-468c-ac45-06eb7627c85d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-7b38deda-6289-468c-ac45-06eb7627c85d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:24.599817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:24.601624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:24.602790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:24.602790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:24.602790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:24.602790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:24.602790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:24.602790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:24.602790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:24.602790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:24.602790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:24.602790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:24.608463 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 406/798] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jun 8 01:52:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:24.610751 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38deda-6289-468c-ac45-06eb7627c85d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:24.637876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 393/799] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:24.639452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c5095-28b9-476c-826e-655ffdc0121e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:24.690176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:24.744528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38deda-6289-468c-ac45-06eb7627c85d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:24.750761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c5095-28b9-476c-826e-655ffdc0121e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:24.783894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1004f39-b3fe-44be-8c3a-f9add36389d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:24.785046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1004f39-b3fe-44be-8c3a-f9add36389d2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.785786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1004f39-b3fe-44be-8c3a-f9add36389d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.786293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1004f39-b3fe-44be-8c3a-f9add36389d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.788099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa343e-ff90-435e-8eb2-b196c2f7d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:24.789072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa343e-ff90-435e-8eb2-b196c2f7d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.789445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa343e-ff90-435e-8eb2-b196c2f7d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.789827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa343e-ff90-435e-8eb2-b196c2f7d0e2 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.790920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa343e-ff90-435e-8eb2-b196c2f7d0e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:24.791170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa343e-ff90-435e-8eb2-b196c2f7d0e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:24.792738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa343e-ff90-435e-8eb2-b196c2f7d0e2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:24.796252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1004f39-b3fe-44be-8c3a-f9add36389d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:24.797903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1004f39-b3fe-44be-8c3a-f9add36389d2 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:24.800728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-e1004f39-b3fe-44be-8c3a-f9add36389d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:24.814637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:24.815572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:24.816659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:24.816659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:24.816659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:24.816659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:24.816659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:24.816659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:24.816659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:24.816659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:24.817096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 394/800] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jun 8 01:52:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:24.817819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa343e-ff90-435e-8eb2-b196c2f7d0e2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:24.820121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 407/801] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:24.821816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1004f39-b3fe-44be-8c3a-f9add36389d2 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:24.938162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa343e-ff90-435e-8eb2-b196c2f7d0e2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:24.940751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:24.942123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1004f39-b3fe-44be-8c3a-f9add36389d2 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:24.975456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42815686-491b-47a4-8f00-183b27d9439e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:24.976434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42815686-491b-47a4-8f00-183b27d9439e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.976671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42815686-491b-47a4-8f00-183b27d9439e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.976951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42815686-491b-47a4-8f00-183b27d9439e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.977664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42815686-491b-47a4-8f00-183b27d9439e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:24.979236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42815686-491b-47a4-8f00-183b27d9439e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:24.982477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d758dd9-6694-4260-9838-8115aca3f40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:24.983481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d758dd9-6694-4260-9838-8115aca3f40e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:24.983967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d758dd9-6694-4260-9838-8115aca3f40e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:24.984498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d758dd9-6694-4260-9838-8115aca3f40e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-42815686-491b-47a4-8f00-183b27d9439e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:24.992619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:24.993578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:24.994415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 395/802] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Jun 8 01:52:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:24.995267 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42815686-491b-47a4-8f00-183b27d9439e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:25.065346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:25.069453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d758dd9-6694-4260-9838-8115aca3f40e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:25.071974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d758dd9-6694-4260-9838-8115aca3f40e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:25.102328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42815686-491b-47a4-8f00-183b27d9439e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:25.138374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e35cbd-03e4-47d6-bbe2-616f6bea4007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:25.138971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e35cbd-03e4-47d6-bbe2-616f6bea4007 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.139262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e35cbd-03e4-47d6-bbe2-616f6bea4007 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.139543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e35cbd-03e4-47d6-bbe2-616f6bea4007 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.140329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e35cbd-03e4-47d6-bbe2-616f6bea4007 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:25.140672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e35cbd-03e4-47d6-bbe2-616f6bea4007 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:25.142238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e35cbd-03e4-47d6-bbe2-616f6bea4007 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:25.157055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 396/803] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:25.158035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e35cbd-03e4-47d6-bbe2-616f6bea4007 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:25.194605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:25.198235 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 408/804] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:52:25.199263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d758dd9-6694-4260-9838-8115aca3f40e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:25.283557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e35cbd-03e4-47d6-bbe2-616f6bea4007 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:25.313122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:25.317039 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d758dd9-6694-4260-9838-8115aca3f40e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:25.323215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10c580-4cf0-4a6a-acdb-d84109fb59be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:25.323968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10c580-4cf0-4a6a-acdb-d84109fb59be None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.324255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10c580-4cf0-4a6a-acdb-d84109fb59be None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.324608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10c580-4cf0-4a6a-acdb-d84109fb59be None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.325333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10c580-4cf0-4a6a-acdb-d84109fb59be None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:25.325646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10c580-4cf0-4a6a-acdb-d84109fb59be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:25.327270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10c580-4cf0-4a6a-acdb-d84109fb59be None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:25.333233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 397/805] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jun 8 01:52:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:25.334337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10c580-4cf0-4a6a-acdb-d84109fb59be None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:25.361290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ebdd8-37f2-4d39-bfd8-0310626e7b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:25.362287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ebdd8-37f2-4d39-bfd8-0310626e7b21 None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.362639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ebdd8-37f2-4d39-bfd8-0310626e7b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.362980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ebdd8-37f2-4d39-bfd8-0310626e7b21 None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/fc299258fe814636880cf595fc2904b4/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.365191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ebdd8-37f2-4d39-bfd8-0310626e7b21 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=fc299258fe814636880cf595fc2904b4, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:25.367191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ebdd8-37f2-4d39-bfd8-0310626e7b21 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:25.383580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 409/806] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:52:25 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/fc299258fe814636880cf595fc2904b4/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:52:25.384779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ebdd8-37f2-4d39-bfd8-0310626e7b21 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:25.454405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:25.464541 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10c580-4cf0-4a6a-acdb-d84109fb59be None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:25.504583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e6920-ef20-46d5-b99c-16fe6ab174e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:25.505674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e6920-ef20-46d5-b99c-16fe6ab174e0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.505788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e6920-ef20-46d5-b99c-16fe6ab174e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.506054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e6920-ef20-46d5-b99c-16fe6ab174e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.513226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ebdd8-37f2-4d39-bfd8-0310626e7b21 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:25.518412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e6920-ef20-46d5-b99c-16fe6ab174e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:25.519894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e6920-ef20-46d5-b99c-16fe6ab174e0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-6d2e6920-ef20-46d5-b99c-16fe6ab174e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:25.540259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:25.541312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:25.542239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:25.542239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:25.542239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:25.542239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:25.546202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 398/807] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:25.547549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e6920-ef20-46d5-b99c-16fe6ab174e0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:25.557336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ee096-4306-4b55-bcfd-1aad8a051d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:25.558391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ee096-4306-4b55-bcfd-1aad8a051d99 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.558892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ee096-4306-4b55-bcfd-1aad8a051d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.559404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ee096-4306-4b55-bcfd-1aad8a051d99 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.560377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ee096-4306-4b55-bcfd-1aad8a051d99 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:25.561063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ee096-4306-4b55-bcfd-1aad8a051d99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:25.562780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ee096-4306-4b55-bcfd-1aad8a051d99 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:25.570095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 410/808] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Jun 8 01:52:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:25.577681 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ee096-4306-4b55-bcfd-1aad8a051d99 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:25.615286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:25.668281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e6920-ef20-46d5-b99c-16fe6ab174e0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:25.707187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc6a24-9494-43cc-9f6f-d7ccbc62dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:25.707735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ee096-4306-4b55-bcfd-1aad8a051d99 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:25.707848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc6a24-9494-43cc-9f6f-d7ccbc62dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.708068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc6a24-9494-43cc-9f6f-d7ccbc62dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.708352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc6a24-9494-43cc-9f6f-d7ccbc62dcb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.709112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc6a24-9494-43cc-9f6f-d7ccbc62dcb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:25.709491 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc6a24-9494-43cc-9f6f-d7ccbc62dcb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:25.711074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc6a24-9494-43cc-9f6f-d7ccbc62dcb4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:25.723474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 399/809] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:25.725888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bcd68-aced-4008-8ba8-842b770a7280 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.726172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bcd68-aced-4008-8ba8-842b770a7280 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.726448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bcd68-aced-4008-8ba8-842b770a7280 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.727836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 400/810] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:25.733295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bcd68-aced-4008-8ba8-842b770a7280 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:25.738447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:25.752173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562f3bb-8f9c-4fb2-995b-cfe3e351fd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:25.752893 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562f3bb-8f9c-4fb2-995b-cfe3e351fd3a None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.753179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562f3bb-8f9c-4fb2-995b-cfe3e351fd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.753463 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562f3bb-8f9c-4fb2-995b-cfe3e351fd3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.754209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562f3bb-8f9c-4fb2-995b-cfe3e351fd3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:25.755625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562f3bb-8f9c-4fb2-995b-cfe3e351fd3a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:25.787042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 411/811] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jun 8 01:52:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:25.790979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed67c-c518-400e-9e1f-be4dbf2990ba None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.792052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed67c-c518-400e-9e1f-be4dbf2990ba None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.794083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed67c-c518-400e-9e1f-be4dbf2990ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.814638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e04ed67c-c518-400e-9e1f-be4dbf2990ba None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:25.815883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e04ed67c-c518-400e-9e1f-be4dbf2990ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:25.898314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bcd68-aced-4008-8ba8-842b770a7280 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:25.914954 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 412/812] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:25.916321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69d699-7a7e-482a-b8c7-f1cdccfeec6f None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.916703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69d699-7a7e-482a-b8c7-f1cdccfeec6f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.916985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69d699-7a7e-482a-b8c7-f1cdccfeec6f None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.922856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 413/813] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:25.924116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95fb1-ec84-417e-9c8d-3f1faa037671 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.924424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95fb1-ec84-417e-9c8d-3f1faa037671 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.924710 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e95fb1-ec84-417e-9c8d-3f1faa037671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.948168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-86e95fb1-ec84-417e-9c8d-3f1faa037671 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:25.949416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-86e95fb1-ec84-417e-9c8d-3f1faa037671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:25.953761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:25.956939 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488ee0f-b75a-4924-b881-5fe70915603b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:25.958240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488ee0f-b75a-4924-b881-5fe70915603b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:25.958543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488ee0f-b75a-4924-b881-5fe70915603b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:25.959956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488ee0f-b75a-4924-b881-5fe70915603b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:25.969005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488ee0f-b75a-4924-b881-5fe70915603b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:25.970581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488ee0f-b75a-4924-b881-5fe70915603b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0488ee0f-b75a-4924-b881-5fe70915603b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:25.986849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:25.987813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:25.988725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:25.988725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:25.988725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:25.988725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:25.988725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:25.988725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:25.988725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:25.988725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:25.988725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 401/814] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:25.989687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488ee0f-b75a-4924-b881-5fe70915603b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:26.046458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 414/815] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:26.047625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e95fb1-ec84-417e-9c8d-3f1faa037671 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:26.059124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:26.105244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488ee0f-b75a-4924-b881-5fe70915603b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:26.154706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95509aa3-2feb-4688-b3af-ed4e4e568146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:26.156475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95509aa3-2feb-4688-b3af-ed4e4e568146 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:26.156972 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95509aa3-2feb-4688-b3af-ed4e4e568146 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:26.157720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95509aa3-2feb-4688-b3af-ed4e4e568146 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:26.168964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e95fb1-ec84-417e-9c8d-3f1faa037671 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:26.173092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95509aa3-2feb-4688-b3af-ed4e4e568146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-95509aa3-2feb-4688-b3af-ed4e4e568146 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:26.182803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:26.183724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:26.184617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:26.187253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 402/816] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:26.187793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95509aa3-2feb-4688-b3af-ed4e4e568146 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:26.212292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ee206-7442-4974-ba0b-eaa75678d245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:26.212923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ee206-7442-4974-ba0b-eaa75678d245 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:26.213208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ee206-7442-4974-ba0b-eaa75678d245 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:26.213486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ee206-7442-4974-ba0b-eaa75678d245 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:26.214340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ee206-7442-4974-ba0b-eaa75678d245 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:26.214669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ee206-7442-4974-ba0b-eaa75678d245 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:26.216165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ee206-7442-4974-ba0b-eaa75678d245 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:26.226347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:26.234500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 415/817] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:26.235682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ee206-7442-4974-ba0b-eaa75678d245 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:26.314077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95509aa3-2feb-4688-b3af-ed4e4e568146 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:26.343211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:26.364726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e98fb-656e-458e-8c54-aac2345f8f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:26.365670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ee206-7442-4974-ba0b-eaa75678d245 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:26.366567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e98fb-656e-458e-8c54-aac2345f8f59 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:26.367053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e98fb-656e-458e-8c54-aac2345f8f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:26.367768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e98fb-656e-458e-8c54-aac2345f8f59 None admin] PATH_INFO: `/v3/services/network` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:26.369607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e98fb-656e-458e-8c54-aac2345f8f59 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:26.371214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e98fb-656e-458e-8c54-aac2345f8f59 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5d4e98fb-656e-458e-8c54-aac2345f8f59 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:26.388502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:26.389543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 08 01:52:26.390372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:26.393074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 403/818] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:26 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:26.394007 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e98fb-656e-458e-8c54-aac2345f8f59 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:26.411615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028fa40-3c59-4dee-964e-cf664abffa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:26.413046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028fa40-3c59-4dee-964e-cf664abffa98 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:26.413666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028fa40-3c59-4dee-964e-cf664abffa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:26.414053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028fa40-3c59-4dee-964e-cf664abffa98 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:26.415022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028fa40-3c59-4dee-964e-cf664abffa98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:26.415456 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028fa40-3c59-4dee-964e-cf664abffa98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:26.417091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028fa40-3c59-4dee-964e-cf664abffa98 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:26.426795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 416/819] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:52:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:26.428196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028fa40-3c59-4dee-964e-cf664abffa98 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:26.500276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:26.512117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e98fb-656e-458e-8c54-aac2345f8f59 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:26.543601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028fa40-3c59-4dee-964e-cf664abffa98 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:26.547680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d70d05-bf68-43d8-b91a-2328704d6247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:26.548347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d70d05-bf68-43d8-b91a-2328704d6247 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:26.548671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d70d05-bf68-43d8-b91a-2328704d6247 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:26.548953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d70d05-bf68-43d8-b91a-2328704d6247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:26.549783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d70d05-bf68-43d8-b91a-2328704d6247 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:26.550099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d70d05-bf68-43d8-b91a-2328704d6247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:26.551535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d70d05-bf68-43d8-b91a-2328704d6247 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:26.558519 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 404/820] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:26 2022] GET /identity/v3/role_assignments?user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:26.559327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d70d05-bf68-43d8-b91a-2328704d6247 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:26.602993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf03de-f2ae-4fcf-ae86-52111bcd3e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:26.603679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf03de-f2ae-4fcf-ae86-52111bcd3e62 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:26.603965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf03de-f2ae-4fcf-ae86-52111bcd3e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:26.604243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf03de-f2ae-4fcf-ae86-52111bcd3e62 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:26.605098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf03de-f2ae-4fcf-ae86-52111bcd3e62 None admin] RBAC: Adding query filter params (name=network) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:26.605411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf03de-f2ae-4fcf-ae86-52111bcd3e62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:26.606912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf03de-f2ae-4fcf-ae86-52111bcd3e62 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:26.617297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:26.618027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 417/821] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:26.624147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf03de-f2ae-4fcf-ae86-52111bcd3e62 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:26.670955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d70d05-bf68-43d8-b91a-2328704d6247 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:26.718576 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d4a0b-74fa-444b-85c0-c385c1846e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:26.719405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d4a0b-74fa-444b-85c0-c385c1846e5b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:26.719689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d4a0b-74fa-444b-85c0-c385c1846e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:26.719965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d4a0b-74fa-444b-85c0-c385c1846e5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:26.742770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d4a0b-74fa-444b-85c0-c385c1846e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:26.744283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d4a0b-74fa-444b-85c0-c385c1846e5b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-1e4d4a0b-74fa-444b-85c0-c385c1846e5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:26.774600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:26.776669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:26.778737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:26.778737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:26.778737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:26.778737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:26.778737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 405/822] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:26.792859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d4a0b-74fa-444b-85c0-c385c1846e5b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:26.859470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf03de-f2ae-4fcf-ae86-52111bcd3e62 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:26.899666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:26.911543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5045c0-8c6f-4fd8-810a-5846791374d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:26.914749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5045c0-8c6f-4fd8-810a-5846791374d6 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:26.915196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5045c0-8c6f-4fd8-810a-5846791374d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:26.915813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5045c0-8c6f-4fd8-810a-5846791374d6 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:26.918125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5045c0-8c6f-4fd8-810a-5846791374d6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:26.919390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5045c0-8c6f-4fd8-810a-5846791374d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:26.920975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5045c0-8c6f-4fd8-810a-5846791374d6 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:26.932410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 418/823] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:26.938969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5045c0-8c6f-4fd8-810a-5846791374d6 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:26.957664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:26.962487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d4a0b-74fa-444b-85c0-c385c1846e5b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:26.999828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a804fe0-3cf2-4a3d-8c6b-af8a08047568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:27.000553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a804fe0-3cf2-4a3d-8c6b-af8a08047568 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:27.000837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a804fe0-3cf2-4a3d-8c6b-af8a08047568 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:27.001112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a804fe0-3cf2-4a3d-8c6b-af8a08047568 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:27.001831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a804fe0-3cf2-4a3d-8c6b-af8a08047568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:27.002139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a804fe0-3cf2-4a3d-8c6b-af8a08047568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:27.003510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a804fe0-3cf2-4a3d-8c6b-af8a08047568 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:27.018432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 406/824] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:27.023668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a804fe0-3cf2-4a3d-8c6b-af8a08047568 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:27.069886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5045c0-8c6f-4fd8-810a-5846791374d6 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:27.116910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412bbf-2d10-48b8-a203-ee538e241b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:27.118358 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412bbf-2d10-48b8-a203-ee538e241b45 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:27.119084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412bbf-2d10-48b8-a203-ee538e241b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:27.119537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9412bbf-2d10-48b8-a203-ee538e241b45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:27.120508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9412bbf-2d10-48b8-a203-ee538e241b45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:27.122846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9412bbf-2d10-48b8-a203-ee538e241b45 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:27.151779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a804fe0-3cf2-4a3d-8c6b-af8a08047568 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:27.154844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 419/825] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jun 8 01:52:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:27.231621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2929f-d4bf-4480-8acc-4fc10ba0aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:27.232492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2929f-d4bf-4480-8acc-4fc10ba0aadc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:27.232779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2929f-d4bf-4480-8acc-4fc10ba0aadc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:27.233053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2929f-d4bf-4480-8acc-4fc10ba0aadc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:27.246553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2929f-d4bf-4480-8acc-4fc10ba0aadc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:27.248049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2929f-d4bf-4480-8acc-4fc10ba0aadc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-51d2929f-d4bf-4480-8acc-4fc10ba0aadc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:27.274820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:27.276056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:27.277444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 407/826] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jun 8 01:52:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:27.281879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412bbf-2d10-48b8-a203-ee538e241b45 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:27.385560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:27.456748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9412bbf-2d10-48b8-a203-ee538e241b45 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:27.495010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e143b-354b-44b5-a425-7fc8d2865acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:27.495696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e143b-354b-44b5-a425-7fc8d2865acc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:27.495984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e143b-354b-44b5-a425-7fc8d2865acc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:27.496268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e143b-354b-44b5-a425-7fc8d2865acc None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:27.497149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e143b-354b-44b5-a425-7fc8d2865acc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:27.497466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e143b-354b-44b5-a425-7fc8d2865acc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:27.498911 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e143b-354b-44b5-a425-7fc8d2865acc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:27.518359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 420/827] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jun 8 01:52:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:27.524084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2929f-d4bf-4480-8acc-4fc10ba0aadc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:27.547474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:27.652514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2929f-d4bf-4480-8acc-4fc10ba0aadc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:27.694744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe8284-ab4a-4ebf-83e2-0c6229d3aed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:27.695617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe8284-ab4a-4ebf-83e2-0c6229d3aed9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:27.695824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe8284-ab4a-4ebf-83e2-0c6229d3aed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:27.696103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe8284-ab4a-4ebf-83e2-0c6229d3aed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:27.706773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe8284-ab4a-4ebf-83e2-0c6229d3aed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:27.708315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe8284-ab4a-4ebf-83e2-0c6229d3aed9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b6fe8284-ab4a-4ebf-83e2-0c6229d3aed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:27.728743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:27.729687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:27.730879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:27.730879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:27.730879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:27.730879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:27.730879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 408/828] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:27.736001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e143b-354b-44b5-a425-7fc8d2865acc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:27.851659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e143b-354b-44b5-a425-7fc8d2865acc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:27.854179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:27.890598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e1f0f-8f75-4134-8bb0-7e4187bf3427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:27.891320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e1f0f-8f75-4134-8bb0-7e4187bf3427 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:27.891611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e1f0f-8f75-4134-8bb0-7e4187bf3427 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:27.891888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e1f0f-8f75-4134-8bb0-7e4187bf3427 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:27.892639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e1f0f-8f75-4134-8bb0-7e4187bf3427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:27.892945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e1f0f-8f75-4134-8bb0-7e4187bf3427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:27.894327 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e1f0f-8f75-4134-8bb0-7e4187bf3427 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:27.907565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 421/829] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:27.913607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe8284-ab4a-4ebf-83e2-0c6229d3aed9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:27.993238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:28.008716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68d9cb-54c6-40f3-8199-e9af696c180d None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.008716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68d9cb-54c6-40f3-8199-e9af696c180d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.008716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68d9cb-54c6-40f3-8199-e9af696c180d None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.010887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 422/830] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:28.016788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc05a48-9003-46b7-93f0-a77404c6723d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.017110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc05a48-9003-46b7-93f0-a77404c6723d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.017360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc05a48-9003-46b7-93f0-a77404c6723d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.033222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe8284-ab4a-4ebf-83e2-0c6229d3aed9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:28.035512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-efc05a48-9003-46b7-93f0-a77404c6723d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:28.036743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-efc05a48-9003-46b7-93f0-a77404c6723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:28.072894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1ef1a-ecb6-4a54-81f9-ce74cee11f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:28.073697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1ef1a-ecb6-4a54-81f9-ce74cee11f7c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.073979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1ef1a-ecb6-4a54-81f9-ce74cee11f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.074251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1ef1a-ecb6-4a54-81f9-ce74cee11f7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.081332 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1ef1a-ecb6-4a54-81f9-ce74cee11f7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:28.082785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1ef1a-ecb6-4a54-81f9-ce74cee11f7c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-48c1ef1a-ecb6-4a54-81f9-ce74cee11f7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:28.097607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:28.100102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:28.102488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:28.102488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:28.102488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:28.102488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:28.102488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:28.102488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:28.102488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:28.102488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:28.102488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 409/831] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:28.106713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1ef1a-ecb6-4a54-81f9-ce74cee11f7c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:28.117974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 423/832] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:28.141870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc05a48-9003-46b7-93f0-a77404c6723d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:28.222042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1ef1a-ecb6-4a54-81f9-ce74cee11f7c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:28.252498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:28.266082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5088d-9393-44a7-89bd-ad1863db6c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:28.267293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5088d-9393-44a7-89bd-ad1863db6c2f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.267747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5088d-9393-44a7-89bd-ad1863db6c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.268422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:28.268838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5088d-9393-44a7-89bd-ad1863db6c2f None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.271184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5088d-9393-44a7-89bd-ad1863db6c2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:28.272633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5088d-9393-44a7-89bd-ad1863db6c2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:28.275892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5088d-9393-44a7-89bd-ad1863db6c2f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:28.276829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc05a48-9003-46b7-93f0-a77404c6723d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:28.293073 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 410/833] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:28.298567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5088d-9393-44a7-89bd-ad1863db6c2f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:28.315722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886fb03-ed2c-4d70-b630-da386f17f9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:28.316483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886fb03-ed2c-4d70-b630-da386f17f9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.316775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886fb03-ed2c-4d70-b630-da386f17f9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.317050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886fb03-ed2c-4d70-b630-da386f17f9d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.328668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886fb03-ed2c-4d70-b630-da386f17f9d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-e886fb03-ed2c-4d70-b630-da386f17f9d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:28.337529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:28.338526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:28.339432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:28.343601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 424/834] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:28.344478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886fb03-ed2c-4d70-b630-da386f17f9d8 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:28.424605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5088d-9393-44a7-89bd-ad1863db6c2f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:28.454502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886fb03-ed2c-4d70-b630-da386f17f9d8 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:28.460091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20432ce8-0e7b-4427-9c49-425882557e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:28.461019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20432ce8-0e7b-4427-9c49-425882557e0b None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.461304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20432ce8-0e7b-4427-9c49-425882557e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.461863 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20432ce8-0e7b-4427-9c49-425882557e0b None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/a6aa350eb1504e9c9be9c90870779381/roles/32b18412b6c944b8899ef24b902885d2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.477475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20432ce8-0e7b-4427-9c49-425882557e0b None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=a6aa350eb1504e9c9be9c90870779381, role_id=32b18412b6c944b8899ef24b902885d2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:28.478864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20432ce8-0e7b-4427-9c49-425882557e0b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:28.490942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146ea32-6509-49c2-9b1f-4f48f3dd692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:28.491823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146ea32-6509-49c2-9b1f-4f48f3dd692a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.492112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146ea32-6509-49c2-9b1f-4f48f3dd692a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.492477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146ea32-6509-49c2-9b1f-4f48f3dd692a None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.493164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146ea32-6509-49c2-9b1f-4f48f3dd692a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:28.500458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146ea32-6509-49c2-9b1f-4f48f3dd692a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:28.500662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 411/835] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:52:28 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/a6aa350eb1504e9c9be9c90870779381/roles/32b18412b6c944b8899ef24b902885d2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 01:52:28.500662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b832b-e817-4699-b699-b132a484ea1d None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.500817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146ea32-6509-49c2-9b1f-4f48f3dd692a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:28.500905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b832b-e817-4699-b699-b132a484ea1d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.500905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b832b-e817-4699-b699-b132a484ea1d None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.500905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 412/836] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:28.502352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b82e2-baec-42c5-b53c-59a32650aee7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.502911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b82e2-baec-42c5-b53c-59a32650aee7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.502911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b82e2-baec-42c5-b53c-59a32650aee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.511302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 425/837] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:28.517601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146ea32-6509-49c2-9b1f-4f48f3dd692a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:28.522097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e66b82e2-baec-42c5-b53c-59a32650aee7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:28.523320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e66b82e2-baec-42c5-b53c-59a32650aee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:28.560608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:28.588865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:28.616429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 413/838] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:28.637448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b82e2-baec-42c5-b53c-59a32650aee7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:28.674245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146ea32-6509-49c2-9b1f-4f48f3dd692a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:28.718728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86ceba-1bb2-401b-ae54-882827da1d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:28.719533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86ceba-1bb2-401b-ae54-882827da1d6b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.719818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86ceba-1bb2-401b-ae54-882827da1d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.720097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86ceba-1bb2-401b-ae54-882827da1d6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.737081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86ceba-1bb2-401b-ae54-882827da1d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:28.739443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86ceba-1bb2-401b-ae54-882827da1d6b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-5b86ceba-1bb2-401b-ae54-882827da1d6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:28.762269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:28.763216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:28.764122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:28.764122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:28.764122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:28.764122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:28.764801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b82e2-baec-42c5-b53c-59a32650aee7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:28.766851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 426/839] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:28.773533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86ceba-1bb2-401b-ae54-882827da1d6b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:28.805603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8533f0d-10ce-4734-9568-905c7db9825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:28.806493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8533f0d-10ce-4734-9568-905c7db9825a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.806706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8533f0d-10ce-4734-9568-905c7db9825a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.806987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8533f0d-10ce-4734-9568-905c7db9825a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.807697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8533f0d-10ce-4734-9568-905c7db9825a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:28.809154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8533f0d-10ce-4734-9568-905c7db9825a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a8533f0d-10ce-4734-9568-905c7db9825a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:28.821991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:28.823099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:28.823933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 414/840] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:52:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:28.829464 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8533f0d-10ce-4734-9568-905c7db9825a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:28.852575 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:28.863893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:28.889353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86ceba-1bb2-401b-ae54-882827da1d6b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:28.935265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864ca72-f440-4b4d-aab5-d2e06ace32ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:28.936791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864ca72-f440-4b4d-aab5-d2e06ace32ca None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.937281 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864ca72-f440-4b4d-aab5-d2e06ace32ca None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.937769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864ca72-f440-4b4d-aab5-d2e06ace32ca None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.939177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864ca72-f440-4b4d-aab5-d2e06ace32ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:28.939674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864ca72-f440-4b4d-aab5-d2e06ace32ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:28.941520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864ca72-f440-4b4d-aab5-d2e06ace32ca None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:28.942105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8533f0d-10ce-4734-9568-905c7db9825a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:28.962859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 427/841] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:28.963829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864ca72-f440-4b4d-aab5-d2e06ace32ca None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:28.985568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbede1e3-7e33-49fe-aad0-c2ead9ed82e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:28.986259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbede1e3-7e33-49fe-aad0-c2ead9ed82e7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:28.986786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbede1e3-7e33-49fe-aad0-c2ead9ed82e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:28.986902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbede1e3-7e33-49fe-aad0-c2ead9ed82e7 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:28.987609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbede1e3-7e33-49fe-aad0-c2ead9ed82e7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:28.987873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbede1e3-7e33-49fe-aad0-c2ead9ed82e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:28.989366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbede1e3-7e33-49fe-aad0-c2ead9ed82e7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:28.995098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 415/842] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:52:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:29.001123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbede1e3-7e33-49fe-aad0-c2ead9ed82e7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:29.106585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864ca72-f440-4b4d-aab5-d2e06ace32ca None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:29.127906 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:29.135330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbede1e3-7e33-49fe-aad0-c2ead9ed82e7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:29.138329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:29.147932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d78266-1fae-4eba-83cc-7ea868a15c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:29.149049 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d78266-1fae-4eba-83cc-7ea868a15c1d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:29.149540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d78266-1fae-4eba-83cc-7ea868a15c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:29.150051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d78266-1fae-4eba-83cc-7ea868a15c1d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:29.158431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d78266-1fae-4eba-83cc-7ea868a15c1d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:29.160164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d78266-1fae-4eba-83cc-7ea868a15c1d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:29.172999 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263d6b4-3514-4dd8-9202-4fa9ad4f3f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:29.173667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263d6b4-3514-4dd8-9202-4fa9ad4f3f96 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:29.173952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263d6b4-3514-4dd8-9202-4fa9ad4f3f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:29.174224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263d6b4-3514-4dd8-9202-4fa9ad4f3f96 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:29.174931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263d6b4-3514-4dd8-9202-4fa9ad4f3f96 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:29.175232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263d6b4-3514-4dd8-9202-4fa9ad4f3f96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:29.176626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263d6b4-3514-4dd8-9202-4fa9ad4f3f96 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-12d78266-1fae-4eba-83cc-7ea868a15c1d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:29.178512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:29.179417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:29.180375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:29.180375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:29.180375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:29.180375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:29.180375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:29.180375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:29.180375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:29.180375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:29.180375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 08 01:52:29.180375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:29.188449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 416/843] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:52:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:29.188644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 428/844] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:29.195474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263d6b4-3514-4dd8-9202-4fa9ad4f3f96 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:29.315886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263d6b4-3514-4dd8-9202-4fa9ad4f3f96 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:29.354746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f6643-e0c5-4f30-8013-0bdcc45b4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:29.355446 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f6643-e0c5-4f30-8013-0bdcc45b4cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:29.355734 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f6643-e0c5-4f30-8013-0bdcc45b4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:29.356013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f6643-e0c5-4f30-8013-0bdcc45b4cd0 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:29.356904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f6643-e0c5-4f30-8013-0bdcc45b4cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:29.357217 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f6643-e0c5-4f30-8013-0bdcc45b4cd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:29.358668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f6643-e0c5-4f30-8013-0bdcc45b4cd0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:29.380105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 417/845] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Jun 8 01:52:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:29.385495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d78266-1fae-4eba-83cc-7ea868a15c1d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:29.430331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73165f5-d06c-42de-998c-d972e7385b98 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:29.430648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73165f5-d06c-42de-998c-d972e7385b98 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:29.430927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73165f5-d06c-42de-998c-d972e7385b98 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:29.432547 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 418/846] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:29.437702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e155e-b2f3-443e-aa1b-58a9489cf5d5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:29.437702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e155e-b2f3-443e-aa1b-58a9489cf5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:29.438000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e155e-b2f3-443e-aa1b-58a9489cf5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:29.457022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-974e155e-b2f3-443e-aa1b-58a9489cf5d5 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:29.458215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-974e155e-b2f3-443e-aa1b-58a9489cf5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:29.476426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:29.506186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d78266-1fae-4eba-83cc-7ea868a15c1d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:29.544419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 419/847] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:29.557564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e5db5-78eb-4ca4-be54-f798abbb7d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:29.558218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e5db5-78eb-4ca4-be54-f798abbb7d8b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:29.558497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e5db5-78eb-4ca4-be54-f798abbb7d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:29.558771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e5db5-78eb-4ca4-be54-f798abbb7d8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:29.566120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e155e-b2f3-443e-aa1b-58a9489cf5d5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:29.641170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:29.659052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e5db5-78eb-4ca4-be54-f798abbb7d8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:29.661591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e5db5-78eb-4ca4-be54-f798abbb7d8b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:29.679343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e155e-b2f3-443e-aa1b-58a9489cf5d5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:29.721292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324534f3-0593-4df0-8ba8-5c578321b570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:29.722105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324534f3-0593-4df0-8ba8-5c578321b570 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:29.722383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324534f3-0593-4df0-8ba8-5c578321b570 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:29.722650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324534f3-0593-4df0-8ba8-5c578321b570 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:29.723294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324534f3-0593-4df0-8ba8-5c578321b570 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:29.724730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324534f3-0593-4df0-8ba8-5c578321b570 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-324534f3-0593-4df0-8ba8-5c578321b570 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:29.738536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:29.739513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 08 01:52:29.740921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:29.744751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 420/848] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Jun 8 01:52:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:29.746254 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324534f3-0593-4df0-8ba8-5c578321b570 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:29.764795 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:29.799257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 429/849] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:52:29.800631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e5db5-78eb-4ca4-be54-f798abbb7d8b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:29.871605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324534f3-0593-4df0-8ba8-5c578321b570 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:29.900854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:29.914834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e5db5-78eb-4ca4-be54-f798abbb7d8b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:29.928989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a08ee7-2d02-4c2c-8b36-d7cc486850c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:29.929987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a08ee7-2d02-4c2c-8b36-d7cc486850c6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:29.930434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a08ee7-2d02-4c2c-8b36-d7cc486850c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:29.930872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a08ee7-2d02-4c2c-8b36-d7cc486850c6 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:29.931842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a08ee7-2d02-4c2c-8b36-d7cc486850c6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:29.932309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a08ee7-2d02-4c2c-8b36-d7cc486850c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:29.933960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a08ee7-2d02-4c2c-8b36-d7cc486850c6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:29.939606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 421/850] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:29.945866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a08ee7-2d02-4c2c-8b36-d7cc486850c6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:29.957881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f239b0-8785-434c-bbf5-4c6e81010d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:29.959422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f239b0-8785-434c-bbf5-4c6e81010d13 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:29.959768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f239b0-8785-434c-bbf5-4c6e81010d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:29.960054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f239b0-8785-434c-bbf5-4c6e81010d13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:29.973927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f239b0-8785-434c-bbf5-4c6e81010d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:29.976011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f239b0-8785-434c-bbf5-4c6e81010d13 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-a7f239b0-8785-434c-bbf5-4c6e81010d13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:30.000363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:30.001316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:30.002205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:30.002205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:30.002205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:30.002205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:30.002205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 430/851] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:30.007770 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f239b0-8785-434c-bbf5-4c6e81010d13 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:30.056778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:30.090283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a08ee7-2d02-4c2c-8b36-d7cc486850c6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:30.118393 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f239b0-8785-434c-bbf5-4c6e81010d13 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:30.136308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53bfb9-8e59-416b-9fb8-742ae3efcd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:30.137173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53bfb9-8e59-416b-9fb8-742ae3efcd0d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:30.137842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53bfb9-8e59-416b-9fb8-742ae3efcd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:30.138124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53bfb9-8e59-416b-9fb8-742ae3efcd0d None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:30.138831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53bfb9-8e59-416b-9fb8-742ae3efcd0d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:30.139135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53bfb9-8e59-416b-9fb8-742ae3efcd0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:30.140534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53bfb9-8e59-416b-9fb8-742ae3efcd0d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:30.148206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 422/852] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:29 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:30.167431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c1657-c662-420c-94ee-263eadc35a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:30.168125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c1657-c662-420c-94ee-263eadc35a8d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:30.168424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c1657-c662-420c-94ee-263eadc35a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:30.168722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c1657-c662-420c-94ee-263eadc35a8d None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:30.169469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c1657-c662-420c-94ee-263eadc35a8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:30.169778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c1657-c662-420c-94ee-263eadc35a8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:30.171207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c1657-c662-420c-94ee-263eadc35a8d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:30.181908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:30.186330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 431/853] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:30.191916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53bfb9-8e59-416b-9fb8-742ae3efcd0d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:30.318026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53bfb9-8e59-416b-9fb8-742ae3efcd0d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:30.359990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5d05f-6fbd-41de-af2d-b9e4f57a556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:30.360830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5d05f-6fbd-41de-af2d-b9e4f57a556c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:30.361111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5d05f-6fbd-41de-af2d-b9e4f57a556c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:30.361381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5d05f-6fbd-41de-af2d-b9e4f57a556c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:30.368946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5d05f-6fbd-41de-af2d-b9e4f57a556c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:30.370403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5d05f-6fbd-41de-af2d-b9e4f57a556c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:30.373508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-7af5d05f-6fbd-41de-af2d-b9e4f57a556c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:30.388058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:30.389221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:30.390229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:30.390229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:30.390229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:30.390229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:30.390229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:30.390229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:30.390229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:30.390229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:30.393935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 423/854] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:30.399030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c1657-c662-420c-94ee-263eadc35a8d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:30.502437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c1657-c662-420c-94ee-263eadc35a8d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:30.538937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1547167-33fd-4df0-b7db-5f1b88916acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:30.540509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547167-33fd-4df0-b7db-5f1b88916acf None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:30.540994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547167-33fd-4df0-b7db-5f1b88916acf None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:30.541487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547167-33fd-4df0-b7db-5f1b88916acf None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:30.542512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1547167-33fd-4df0-b7db-5f1b88916acf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:30.543054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1547167-33fd-4df0-b7db-5f1b88916acf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:30.544801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1547167-33fd-4df0-b7db-5f1b88916acf None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:30.560772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 432/855] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:30.570502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5d05f-6fbd-41de-af2d-b9e4f57a556c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:30.682914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5d05f-6fbd-41de-af2d-b9e4f57a556c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:30.718541 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aedbe9-1042-4f9c-8790-522522f13503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:30.719579 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aedbe9-1042-4f9c-8790-522522f13503 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:30.720076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aedbe9-1042-4f9c-8790-522522f13503 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:30.720587 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aedbe9-1042-4f9c-8790-522522f13503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:30.721622 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aedbe9-1042-4f9c-8790-522522f13503 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=fc299258fe814636880cf595fc2904b4) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:30.722148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aedbe9-1042-4f9c-8790-522522f13503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:30.723776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aedbe9-1042-4f9c-8790-522522f13503 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:30.731130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 424/856] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:30 2022] GET /identity/v3/role_assignments?user.id=fc299258fe814636880cf595fc2904b4&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:30.889512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ca5f1-7d2a-4392-99a7-1604c69587b4 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:30.889848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ca5f1-7d2a-4392-99a7-1604c69587b4 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:30.890065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ca5f1-7d2a-4392-99a7-1604c69587b4 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:30.891611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 433/857] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:30.897823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faec6e-9eca-479f-a675-f7b8a2ddc636 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:30.898811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faec6e-9eca-479f-a675-f7b8a2ddc636 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:30.899342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faec6e-9eca-479f-a675-f7b8a2ddc636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:30.930607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-56faec6e-9eca-479f-a675-f7b8a2ddc636 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:30.932180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-56faec6e-9eca-479f-a675-f7b8a2ddc636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:30.999286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:31.012509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 425/858] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:31.032872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ca5f1-7d2a-4392-99a7-1604c69587b4 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:31.100324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:31.142232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ca5f1-7d2a-4392-99a7-1604c69587b4 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:31.179405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8938902-dd0f-496f-96b6-15ddfeb47907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:31.180291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8938902-dd0f-496f-96b6-15ddfeb47907 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:31.180616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8938902-dd0f-496f-96b6-15ddfeb47907 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:31.180890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8938902-dd0f-496f-96b6-15ddfeb47907 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:31.193786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8938902-dd0f-496f-96b6-15ddfeb47907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-d8938902-dd0f-496f-96b6-15ddfeb47907 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:31.202075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:31.203102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:31.203953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 434/859] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:52:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:31.210550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faec6e-9eca-479f-a675-f7b8a2ddc636 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:31.316146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faec6e-9eca-479f-a675-f7b8a2ddc636 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:31.352040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f9bca-8f7b-409b-bac5-dbb2c6868124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:31.352759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f9bca-8f7b-409b-bac5-dbb2c6868124 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:31.353044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f9bca-8f7b-409b-bac5-dbb2c6868124 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:31.353408 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f9bca-8f7b-409b-bac5-dbb2c6868124 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:31.354284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f9bca-8f7b-409b-bac5-dbb2c6868124 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:31.354597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f9bca-8f7b-409b-bac5-dbb2c6868124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:31.355997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f9bca-8f7b-409b-bac5-dbb2c6868124 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:31.370279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 426/860] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:52:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:31.373573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8938902-dd0f-496f-96b6-15ddfeb47907 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:31.480641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8938902-dd0f-496f-96b6-15ddfeb47907 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:31.501414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:31.516498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5314402-5048-44ce-8ae2-ea375ae54f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:31.517301 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5314402-5048-44ce-8ae2-ea375ae54f56 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:31.517584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5314402-5048-44ce-8ae2-ea375ae54f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:31.517861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5314402-5048-44ce-8ae2-ea375ae54f56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:31.529363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5314402-5048-44ce-8ae2-ea375ae54f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:31.530839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5314402-5048-44ce-8ae2-ea375ae54f56 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:31.545454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a896cc-32a1-4ac4-badc-e529f52fd976 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:31.546063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a896cc-32a1-4ac4-badc-e529f52fd976 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:31.546577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a896cc-32a1-4ac4-badc-e529f52fd976 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:31.548432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 427/861] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-d5314402-5048-44ce-8ae2-ea375ae54f56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:31.553245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:31.554547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:31.555443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:31.555443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:31.555443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:31.555443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:31.555443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 435/862] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:31.556842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619c7e6-d17a-4344-89fc-d7030249cb00 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:31.557356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619c7e6-d17a-4344-89fc-d7030249cb00 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:31.557850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619c7e6-d17a-4344-89fc-d7030249cb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:31.559829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5314402-5048-44ce-8ae2-ea375ae54f56 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:31.578768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0619c7e6-d17a-4344-89fc-d7030249cb00 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:31.580430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0619c7e6-d17a-4344-89fc-d7030249cb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:31.638221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:31.672417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 428/863] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:31.686431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619c7e6-d17a-4344-89fc-d7030249cb00 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:31.689836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5314402-5048-44ce-8ae2-ea375ae54f56 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:31.727750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b01e6-233d-4f18-809e-a2fa4afa6ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:31.728818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b01e6-233d-4f18-809e-a2fa4afa6ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:31.729423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b01e6-233d-4f18-809e-a2fa4afa6ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:31.730012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b01e6-233d-4f18-809e-a2fa4afa6ce7 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:31.730938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b01e6-233d-4f18-809e-a2fa4afa6ce7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:31.731462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b01e6-233d-4f18-809e-a2fa4afa6ce7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:31.733113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b01e6-233d-4f18-809e-a2fa4afa6ce7 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:31.747029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 436/864] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:31.752835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b01e6-233d-4f18-809e-a2fa4afa6ce7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:31.797795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:31.808590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619c7e6-d17a-4344-89fc-d7030249cb00 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:31.845578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892149c-bcef-432a-a61a-e930f2dd91e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:31.846594 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892149c-bcef-432a-a61a-e930f2dd91e8 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:31.847080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892149c-bcef-432a-a61a-e930f2dd91e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:31.847557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892149c-bcef-432a-a61a-e930f2dd91e8 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:31.848568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892149c-bcef-432a-a61a-e930f2dd91e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:31.850794 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892149c-bcef-432a-a61a-e930f2dd91e8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:31.872910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 429/865] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:52:31 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:52:31.887669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b01e6-233d-4f18-809e-a2fa4afa6ce7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:31.947933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78811d58-6989-4f30-a5d4-0f8e20451ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:31.948812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78811d58-6989-4f30-a5d4-0f8e20451ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:31.949098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78811d58-6989-4f30-a5d4-0f8e20451ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:31.949372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78811d58-6989-4f30-a5d4-0f8e20451ed8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:31.957890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78811d58-6989-4f30-a5d4-0f8e20451ed8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:31.960179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78811d58-6989-4f30-a5d4-0f8e20451ed8 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-78811d58-6989-4f30-a5d4-0f8e20451ed8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:31.977960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:31.978985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:31.979929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:31.979929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:31.979929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:31.979929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:31.979929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:31.979929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:31.979929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:31.979929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:31.979929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:31.979929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:31.983978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 437/866] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jun 8 01:52:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:31.985403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892149c-bcef-432a-a61a-e930f2dd91e8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:32.041032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:32.106963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892149c-bcef-432a-a61a-e930f2dd91e8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:32.143965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d79fc-66bb-4462-a390-1de034b07784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:32.144737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d79fc-66bb-4462-a390-1de034b07784 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:32.145019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d79fc-66bb-4462-a390-1de034b07784 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:32.145342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d79fc-66bb-4462-a390-1de034b07784 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:32.146184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d79fc-66bb-4462-a390-1de034b07784 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:32.146519 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d79fc-66bb-4462-a390-1de034b07784 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:32.148405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d79fc-66bb-4462-a390-1de034b07784 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:32.168347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 430/867] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jun 8 01:52:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:32.173667 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78811d58-6989-4f30-a5d4-0f8e20451ed8 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:32.270465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:32.282984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78811d58-6989-4f30-a5d4-0f8e20451ed8 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:32.321428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f129c1-e141-4541-ad04-abd1006251c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:32.322094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f129c1-e141-4541-ad04-abd1006251c4 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:32.322380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f129c1-e141-4541-ad04-abd1006251c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:32.322660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f129c1-e141-4541-ad04-abd1006251c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:32.394100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f129c1-e141-4541-ad04-abd1006251c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:32.395506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f129c1-e141-4541-ad04-abd1006251c4 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:32.487188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:32.504517 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 438/868] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:32 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 08 01:52:32.507887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d79fc-66bb-4462-a390-1de034b07784 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:32.580200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a3eb3-156c-4430-b0b8-d6818bb027d6 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:32.580731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a3eb3-156c-4430-b0b8-d6818bb027d6 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:32.580833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a3eb3-156c-4430-b0b8-d6818bb027d6 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:32.583149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 439/869] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:32.589959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f35d87-65ac-4c4a-a33f-4c95325ccc6e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:32.590361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f35d87-65ac-4c4a-a33f-4c95325ccc6e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:32.590744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f35d87-65ac-4c4a-a33f-4c95325ccc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:32.608659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-92f35d87-65ac-4c4a-a33f-4c95325ccc6e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:32.610006 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-92f35d87-65ac-4c4a-a33f-4c95325ccc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:32.617991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d79fc-66bb-4462-a390-1de034b07784 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:32.620389 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:32.654770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d27d51-8418-404f-95bd-b37eb09ab6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:32.655495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d27d51-8418-404f-95bd-b37eb09ab6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:32.655775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d27d51-8418-404f-95bd-b37eb09ab6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:32.656047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d27d51-8418-404f-95bd-b37eb09ab6b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:32.667396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d27d51-8418-404f-95bd-b37eb09ab6b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:32.668890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d27d51-8418-404f-95bd-b37eb09ab6b3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-38d27d51-8418-404f-95bd-b37eb09ab6b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:32.687384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:32.688318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:32.689244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:32.689244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:32.689244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:32.689244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:32.689244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 431/870] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:32.692530 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 440/871] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:32.694370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d27d51-8418-404f-95bd-b37eb09ab6b3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:32.715297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f35d87-65ac-4c4a-a33f-4c95325ccc6e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:32.834481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d27d51-8418-404f-95bd-b37eb09ab6b3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:32.851244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f35d87-65ac-4c4a-a33f-4c95325ccc6e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:32.875464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:32.887687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e4e58-85f7-42fc-af7a-dced71c55272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:32.888936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e4e58-85f7-42fc-af7a-dced71c55272 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:32.889679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e4e58-85f7-42fc-af7a-dced71c55272 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:32.890560 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e4e58-85f7-42fc-af7a-dced71c55272 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:32.891927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e4e58-85f7-42fc-af7a-dced71c55272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:32.892710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e4e58-85f7-42fc-af7a-dced71c55272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:32.894494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e4e58-85f7-42fc-af7a-dced71c55272 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:32.896922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28de750-893c-4d12-948d-a1750f0207b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:32.897597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28de750-893c-4d12-948d-a1750f0207b3 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:32.897878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28de750-893c-4d12-948d-a1750f0207b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:32.898156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28de750-893c-4d12-948d-a1750f0207b3 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:32.899442 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28de750-893c-4d12-948d-a1750f0207b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:32.900856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28de750-893c-4d12-948d-a1750f0207b3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:32.917848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 432/872] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:32.928450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 441/873] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:52:32 2022] POST /identity/v3/services => generated 269 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:52:32.928665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e4e58-85f7-42fc-af7a-dced71c55272 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:32.950655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:33.044824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e4e58-85f7-42fc-af7a-dced71c55272 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:33.088498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b6f75-9008-49b3-85e5-e4981a7d4e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:33.089310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6f75-9008-49b3-85e5-e4981a7d4e61 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:33.089589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6f75-9008-49b3-85e5-e4981a7d4e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:33.089860 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6f75-9008-49b3-85e5-e4981a7d4e61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:33.097737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b6f75-9008-49b3-85e5-e4981a7d4e61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:33.099320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b6f75-9008-49b3-85e5-e4981a7d4e61 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-659b6f75-9008-49b3-85e5-e4981a7d4e61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:33.116333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:33.117364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:33.118497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:33.118497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:33.118497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:33.118497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:33.118497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:33.118497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:33.118497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:33.118497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:33.118497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 433/874] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:33.125577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28de750-893c-4d12-948d-a1750f0207b3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:33.184994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952812b0-acaa-424c-b004-e97015e7ea6c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:33.185486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952812b0-acaa-424c-b004-e97015e7ea6c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:33.185771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952812b0-acaa-424c-b004-e97015e7ea6c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:33.187279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 434/875] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:33.194161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f99b1-a2d1-4937-bf92-446217c7a63a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:33.194568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f99b1-a2d1-4937-bf92-446217c7a63a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:33.194920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f99b1-a2d1-4937-bf92-446217c7a63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:33.214703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-037f99b1-a2d1-4937-bf92-446217c7a63a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:33.215904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-037f99b1-a2d1-4937-bf92-446217c7a63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:33.255285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28de750-893c-4d12-948d-a1750f0207b3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:33.305924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 435/876] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:33.309716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b1516-643b-4d82-87f7-3b768f10a1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:33.310400 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b1516-643b-4d82-87f7-3b768f10a1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:33.310688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b1516-643b-4d82-87f7-3b768f10a1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:33.310967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b1516-643b-4d82-87f7-3b768f10a1f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:33.311772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b1516-643b-4d82-87f7-3b768f10a1f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:33.312088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b1516-643b-4d82-87f7-3b768f10a1f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:33.313583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b1516-643b-4d82-87f7-3b768f10a1f2 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:33.328472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f99b1-a2d1-4937-bf92-446217c7a63a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:33.331106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:33.331384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 442/877] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:33.333422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b1516-643b-4d82-87f7-3b768f10a1f2 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:33.376726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:33.450602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f99b1-a2d1-4937-bf92-446217c7a63a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:33.461861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b1516-643b-4d82-87f7-3b768f10a1f2 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:33.487301 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f791-6002-426e-8df4-d206b4fedbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:33.488116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f791-6002-426e-8df4-d206b4fedbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:33.488422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f791-6002-426e-8df4-d206b4fedbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:33.488729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f791-6002-426e-8df4-d206b4fedbe8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:33.489591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f791-6002-426e-8df4-d206b4fedbe8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:33.491539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f791-6002-426e-8df4-d206b4fedbe8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-0137f791-6002-426e-8df4-d206b4fedbe8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:33.504247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:33.505298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:33.506551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:33.508474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 436/878] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:33.512054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f791-6002-426e-8df4-d206b4fedbe8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:33.514040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030cd87-f1e1-4765-98d5-b811c4654152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:33.514767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030cd87-f1e1-4765-98d5-b811c4654152 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:33.515053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030cd87-f1e1-4765-98d5-b811c4654152 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:33.515328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030cd87-f1e1-4765-98d5-b811c4654152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:33.516176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030cd87-f1e1-4765-98d5-b811c4654152 None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=a6aa350eb1504e9c9be9c90870779381) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:33.516686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030cd87-f1e1-4765-98d5-b811c4654152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:33.519173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030cd87-f1e1-4765-98d5-b811c4654152 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:33.532425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 443/879] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:33 2022] GET /identity/v3/role_assignments?user.id=a6aa350eb1504e9c9be9c90870779381&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:33.596822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:33.638106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f791-6002-426e-8df4-d206b4fedbe8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:33.677671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4219e40-002c-4227-bb1d-c1aa06790aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:33.678373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4219e40-002c-4227-bb1d-c1aa06790aab None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:33.678672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4219e40-002c-4227-bb1d-c1aa06790aab None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:33.678936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4219e40-002c-4227-bb1d-c1aa06790aab None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:33.679657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4219e40-002c-4227-bb1d-c1aa06790aab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:33.679962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4219e40-002c-4227-bb1d-c1aa06790aab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:33.681406 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4219e40-002c-4227-bb1d-c1aa06790aab None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:33.687266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 437/880] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jun 8 01:52:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:33.697779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030cd87-f1e1-4765-98d5-b811c4654152 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:33.822552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030cd87-f1e1-4765-98d5-b811c4654152 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:33.825895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:33.866591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604957a-f6d9-48c2-be8c-a6bf39de4b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:33.867655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604957a-f6d9-48c2-be8c-a6bf39de4b07 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:33.868147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604957a-f6d9-48c2-be8c-a6bf39de4b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:33.868662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604957a-f6d9-48c2-be8c-a6bf39de4b07 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:33.869598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604957a-f6d9-48c2-be8c-a6bf39de4b07 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:33.870136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604957a-f6d9-48c2-be8c-a6bf39de4b07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:33.871847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604957a-f6d9-48c2-be8c-a6bf39de4b07 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:33.877723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 444/881] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jun 8 01:52:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:34.287604 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc3fff-6b34-42cc-af6d-c9972442bc8f None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:34.288299 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc3fff-6b34-42cc-af6d-c9972442bc8f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:34.288299 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc3fff-6b34-42cc-af6d-c9972442bc8f None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:34.289687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 438/882] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:34.295128 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2303380-2d4d-429a-9ffa-403c674a6056 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:34.295432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2303380-2d4d-429a-9ffa-403c674a6056 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:34.295719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2303380-2d4d-429a-9ffa-403c674a6056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:34.313511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-f2303380-2d4d-429a-9ffa-403c674a6056 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:34.314725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f2303380-2d4d-429a-9ffa-403c674a6056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:34.385007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 445/883] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:34.407727 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc3fff-6b34-42cc-af6d-c9972442bc8f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:34.514356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc3fff-6b34-42cc-af6d-c9972442bc8f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:34.548329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:34.555120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e3ef7-2095-493e-bfe8-1bc9f6441920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:34.555987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e3ef7-2095-493e-bfe8-1bc9f6441920 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:34.556278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e3ef7-2095-493e-bfe8-1bc9f6441920 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:34.556587 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e3ef7-2095-493e-bfe8-1bc9f6441920 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:34.557293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e3ef7-2095-493e-bfe8-1bc9f6441920 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:34.558688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e3ef7-2095-493e-bfe8-1bc9f6441920 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-af1e3ef7-2095-493e-bfe8-1bc9f6441920 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:34.572130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:34.573173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:34.574027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 439/884] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:52:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:34.581245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2303380-2d4d-429a-9ffa-403c674a6056 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:34.684024 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:34.690040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2303380-2d4d-429a-9ffa-403c674a6056 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:34.726729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8f29c-dc31-4ffd-bc8f-33f2fd490abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:34.727528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8f29c-dc31-4ffd-bc8f-33f2fd490abf None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:34.727720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8f29c-dc31-4ffd-bc8f-33f2fd490abf None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:34.727955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8f29c-dc31-4ffd-bc8f-33f2fd490abf None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:34.728704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8f29c-dc31-4ffd-bc8f-33f2fd490abf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:34.729014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8f29c-dc31-4ffd-bc8f-33f2fd490abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:34.730401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8f29c-dc31-4ffd-bc8f-33f2fd490abf None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:34.735717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 446/885] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:52:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:34.741806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e3ef7-2095-493e-bfe8-1bc9f6441920 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:34.856627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e3ef7-2095-493e-bfe8-1bc9f6441920 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:34.893080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbeecdc-547f-4bd7-8b95-2b3c2bbfe22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:34.893759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbeecdc-547f-4bd7-8b95-2b3c2bbfe22f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:34.894045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbeecdc-547f-4bd7-8b95-2b3c2bbfe22f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:34.894325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbeecdc-547f-4bd7-8b95-2b3c2bbfe22f None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:34.895037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbeecdc-547f-4bd7-8b95-2b3c2bbfe22f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:34.895347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbeecdc-547f-4bd7-8b95-2b3c2bbfe22f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:34.897304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbeecdc-547f-4bd7-8b95-2b3c2bbfe22f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:34.902828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 440/886] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:52:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:34.915549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8f29c-dc31-4ffd-bc8f-33f2fd490abf None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:35.033344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8f29c-dc31-4ffd-bc8f-33f2fd490abf None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:35.069159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fdd68-3a07-4170-8c7e-f2e594b73e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:35.069828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fdd68-3a07-4170-8c7e-f2e594b73e47 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:35.070116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fdd68-3a07-4170-8c7e-f2e594b73e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:35.070396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fdd68-3a07-4170-8c7e-f2e594b73e47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:35.071184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fdd68-3a07-4170-8c7e-f2e594b73e47 None admin] RBAC: Adding query filter params (interface=public, service_id=d5be8c3698e64836a8d1e2748543318e, region_id=RegionOne) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:35.071506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fdd68-3a07-4170-8c7e-f2e594b73e47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:35.072927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fdd68-3a07-4170-8c7e-f2e594b73e47 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:35.080926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 447/887] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jun 8 01:52:34 2022] GET /identity/v3/endpoints?service_id=d5be8c3698e64836a8d1e2748543318e&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:35.087101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbeecdc-547f-4bd7-8b95-2b3c2bbfe22f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:35.158192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:35.203121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbeecdc-547f-4bd7-8b95-2b3c2bbfe22f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:35.244888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b88b35-5893-4764-b1ad-c527b1ed5ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:35.245573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b88b35-5893-4764-b1ad-c527b1ed5ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:35.245857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b88b35-5893-4764-b1ad-c527b1ed5ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:35.246133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b88b35-5893-4764-b1ad-c527b1ed5ca4 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:35.246823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b88b35-5893-4764-b1ad-c527b1ed5ca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:35.248209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b88b35-5893-4764-b1ad-c527b1ed5ca4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:35.254641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 441/888] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jun 8 01:52:35 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 01:52:35.278720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1493f-d5c2-4dd6-9a5b-3b089b822310 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:35.279037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1493f-d5c2-4dd6-9a5b-3b089b822310 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:35.279274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1493f-d5c2-4dd6-9a5b-3b089b822310 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:35.281009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 448/889] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:35.287543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45993a-4b16-46ce-99e2-f6465472feb3 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:35.287873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45993a-4b16-46ce-99e2-f6465472feb3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:35.288723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45993a-4b16-46ce-99e2-f6465472feb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:35.307410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4c45993a-4b16-46ce-99e2-f6465472feb3 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:35.308656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4c45993a-4b16-46ce-99e2-f6465472feb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:35.409436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 442/890] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:35.433189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1493f-d5c2-4dd6-9a5b-3b089b822310 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:35.467307 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:35.558478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1493f-d5c2-4dd6-9a5b-3b089b822310 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:35.597668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a37ad-1f18-48cc-9d96-9ccc7f5f681f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:35.598633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a37ad-1f18-48cc-9d96-9ccc7f5f681f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:35.598916 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a37ad-1f18-48cc-9d96-9ccc7f5f681f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:35.599190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a37ad-1f18-48cc-9d96-9ccc7f5f681f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:35.599967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a37ad-1f18-48cc-9d96-9ccc7f5f681f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:35.601562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a37ad-1f18-48cc-9d96-9ccc7f5f681f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-e36a37ad-1f18-48cc-9d96-9ccc7f5f681f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:35.616459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:35.617917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:35.618938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 449/891] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Jun 8 01:52:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:35.625782 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45993a-4b16-46ce-99e2-f6465472feb3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:35.685479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:35.737637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45993a-4b16-46ce-99e2-f6465472feb3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:35.779006 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47bab9-f70b-4fe2-9e12-39dde654d850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:35.779703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47bab9-f70b-4fe2-9e12-39dde654d850 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:35.780024 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47bab9-f70b-4fe2-9e12-39dde654d850 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:35.780308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47bab9-f70b-4fe2-9e12-39dde654d850 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:35.781090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47bab9-f70b-4fe2-9e12-39dde654d850 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:35.781386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47bab9-f70b-4fe2-9e12-39dde654d850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:35.782833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47bab9-f70b-4fe2-9e12-39dde654d850 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:35.790821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 443/892] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:35.797224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a37ad-1f18-48cc-9d96-9ccc7f5f681f None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:35.898585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:35.917355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a37ad-1f18-48cc-9d96-9ccc7f5f681f None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:35.967589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce6e05-18eb-42b6-b177-d4a566a3dc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:35.976728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce6e05-18eb-42b6-b177-d4a566a3dc6a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:35.976728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce6e05-18eb-42b6-b177-d4a566a3dc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:35.976728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce6e05-18eb-42b6-b177-d4a566a3dc6a None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:35.976728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce6e05-18eb-42b6-b177-d4a566a3dc6a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:35.976728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce6e05-18eb-42b6-b177-d4a566a3dc6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:35.976728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce6e05-18eb-42b6-b177-d4a566a3dc6a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:35.978840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 450/893] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:35.983614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47bab9-f70b-4fe2-9e12-39dde654d850 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:36.104957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47bab9-f70b-4fe2-9e12-39dde654d850 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:36.125402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:36.145322 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec9035-cd1f-43a6-8472-3df8b2bba31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:36.145995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9035-cd1f-43a6-8472-3df8b2bba31b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:36.146327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9035-cd1f-43a6-8472-3df8b2bba31b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:36.147115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9035-cd1f-43a6-8472-3df8b2bba31b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:36.147447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec9035-cd1f-43a6-8472-3df8b2bba31b None admin] RBAC: Adding query filter params (interface=public, service_id=b86f1ba467cc4d5a9ea92c4adaf41164, region_id=RegionOne) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:36.147764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec9035-cd1f-43a6-8472-3df8b2bba31b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:36.149173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec9035-cd1f-43a6-8472-3df8b2bba31b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:36.155482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 444/894] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jun 8 01:52:35 2022] GET /identity/v3/endpoints?service_id=b86f1ba467cc4d5a9ea92c4adaf41164&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:36.161215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce6e05-18eb-42b6-b177-d4a566a3dc6a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:36.163425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c7acb-9fb5-483e-be60-8dff04fbcc90 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:36.163776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c7acb-9fb5-483e-be60-8dff04fbcc90 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:36.164056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c7acb-9fb5-483e-be60-8dff04fbcc90 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:36.165733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 445/895] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:36.171088 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbff9e-9037-490b-a710-fab506b46963 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:36.171372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbff9e-9037-490b-a710-fab506b46963 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:36.171648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbff9e-9037-490b-a710-fab506b46963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:36.190287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-35dbff9e-9037-490b-a710-fab506b46963 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:36.191496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-35dbff9e-9037-490b-a710-fab506b46963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:36.283446 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 446/896] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:36.301361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce6e05-18eb-42b6-b177-d4a566a3dc6a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:36.306191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbff9e-9037-490b-a710-fab506b46963 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:36.361770 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7161a5-fac5-4ee2-ad62-2512e0a30354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:36.363254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7161a5-fac5-4ee2-ad62-2512e0a30354 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:36.363980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7161a5-fac5-4ee2-ad62-2512e0a30354 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:36.364455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7161a5-fac5-4ee2-ad62-2512e0a30354 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:36.365863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7161a5-fac5-4ee2-ad62-2512e0a30354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:36.368195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7161a5-fac5-4ee2-ad62-2512e0a30354 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:36.376717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 451/897] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jun 8 01:52:36 2022] GET /identity/v3/services => generated 1268 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 01:52:36.438427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbff9e-9037-490b-a710-fab506b46963 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:36.452449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:36.482216 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d395f9-baf7-433b-9e85-c53c3b20598c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:36.483584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d395f9-baf7-433b-9e85-c53c3b20598c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:36.484111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d395f9-baf7-433b-9e85-c53c3b20598c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:36.484641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d395f9-baf7-433b-9e85-c53c3b20598c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:36.485077 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d7d86-760b-470c-8e04-aad79a25d736 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:36.485374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d7d86-760b-470c-8e04-aad79a25d736 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:36.485658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d7d86-760b-470c-8e04-aad79a25d736 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:36.487350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 452/898] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:36.493132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2c96c-3740-431a-8611-0da2f649eea2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:36.493434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2c96c-3740-431a-8611-0da2f649eea2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:36.493701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2c96c-3740-431a-8611-0da2f649eea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:36.500502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d395f9-baf7-433b-9e85-c53c3b20598c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-61d395f9-baf7-433b-9e85-c53c3b20598c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:36.510078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:36.511089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:52:36.511938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:36.513960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 447/899] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jun 8 01:52:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:36.518120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-60f2c96c-3740-431a-8611-0da2f649eea2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:36.518120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-60f2c96c-3740-431a-8611-0da2f649eea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:36.521960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d395f9-baf7-433b-9e85-c53c3b20598c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:36.533014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:36.627730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 453/900] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:36.652054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2c96c-3740-431a-8611-0da2f649eea2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:36.659885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d395f9-baf7-433b-9e85-c53c3b20598c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:36.697343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389a9e1-4034-4738-8a7e-9d9ec4c17a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:36.698302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389a9e1-4034-4738-8a7e-9d9ec4c17a08 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:36.698737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389a9e1-4034-4738-8a7e-9d9ec4c17a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:36.699255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389a9e1-4034-4738-8a7e-9d9ec4c17a08 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:36.700264 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389a9e1-4034-4738-8a7e-9d9ec4c17a08 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:36.700855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389a9e1-4034-4738-8a7e-9d9ec4c17a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:36.702475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389a9e1-4034-4738-8a7e-9d9ec4c17a08 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:36.714227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 448/901] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jun 8 01:52:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:36.719772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389a9e1-4034-4738-8a7e-9d9ec4c17a08 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:36.762035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:36.769847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2c96c-3740-431a-8611-0da2f649eea2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:36.813496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c98d2-0c23-4169-9a34-a1c2367f05cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:36.814286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c98d2-0c23-4169-9a34-a1c2367f05cc None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:36.814488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c98d2-0c23-4169-9a34-a1c2367f05cc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:36.814717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c98d2-0c23-4169-9a34-a1c2367f05cc None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:36.815435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c98d2-0c23-4169-9a34-a1c2367f05cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:36.817378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c98d2-0c23-4169-9a34-a1c2367f05cc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:36.836987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 454/902] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:52:36 2022] POST /identity/v3/services => generated 241 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:52:36.856429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389a9e1-4034-4738-8a7e-9d9ec4c17a08 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:36.894798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b22378-509b-447d-a8e0-e90000ca04ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:36.895538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b22378-509b-447d-a8e0-e90000ca04ac None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:36.895799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b22378-509b-447d-a8e0-e90000ca04ac None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:36.896070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b22378-509b-447d-a8e0-e90000ca04ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:36.911350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b22378-509b-447d-a8e0-e90000ca04ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:36.912536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b22378-509b-447d-a8e0-e90000ca04ac None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-33b22378-509b-447d-a8e0-e90000ca04ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:36.932732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:36.933712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:36.934626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:36.934626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:36.934626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:36.934626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:36.934626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 449/903] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:36.941113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c98d2-0c23-4169-9a34-a1c2367f05cc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:36.951760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:37.066595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c98d2-0c23-4169-9a34-a1c2367f05cc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:37.108690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70dc13-6636-4940-9929-95d83c0bd7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:37.109370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70dc13-6636-4940-9929-95d83c0bd7ba None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:37.109657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70dc13-6636-4940-9929-95d83c0bd7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:37.109937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70dc13-6636-4940-9929-95d83c0bd7ba None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:37.110666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70dc13-6636-4940-9929-95d83c0bd7ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:37.110974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70dc13-6636-4940-9929-95d83c0bd7ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:37.112358 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70dc13-6636-4940-9929-95d83c0bd7ba None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:37.125860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 455/904] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:37.131677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b22378-509b-447d-a8e0-e90000ca04ac None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:37.216992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:37.241665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b22378-509b-447d-a8e0-e90000ca04ac None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:37.277396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76571b83-4aa0-4199-94c3-2483ed3dc4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:37.278216 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76571b83-4aa0-4199-94c3-2483ed3dc4dc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:37.278495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76571b83-4aa0-4199-94c3-2483ed3dc4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:37.278766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76571b83-4aa0-4199-94c3-2483ed3dc4dc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:37.287613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76571b83-4aa0-4199-94c3-2483ed3dc4dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:37.289280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76571b83-4aa0-4199-94c3-2483ed3dc4dc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-76571b83-4aa0-4199-94c3-2483ed3dc4dc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:37.305678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:37.307132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:37.308113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 450/905] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jun 8 01:52:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:37.312585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70dc13-6636-4940-9929-95d83c0bd7ba None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:37.394594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:37.431547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70dc13-6636-4940-9929-95d83c0bd7ba None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:37.466300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725e85f-c55a-4d7a-aa90-0affc8f9d6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:37.466968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725e85f-c55a-4d7a-aa90-0affc8f9d6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:37.467296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725e85f-c55a-4d7a-aa90-0affc8f9d6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:37.467520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725e85f-c55a-4d7a-aa90-0affc8f9d6d1 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:37.468351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725e85f-c55a-4d7a-aa90-0affc8f9d6d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:37.468686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725e85f-c55a-4d7a-aa90-0affc8f9d6d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:37.470112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725e85f-c55a-4d7a-aa90-0affc8f9d6d1 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:37.487964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 456/906] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jun 8 01:52:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:37.493854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76571b83-4aa0-4199-94c3-2483ed3dc4dc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:37.603359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76571b83-4aa0-4199-94c3-2483ed3dc4dc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:37.638724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11c35b-c779-41e7-8a59-28441b229e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:37.639402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11c35b-c779-41e7-8a59-28441b229e72 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:37.639684 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11c35b-c779-41e7-8a59-28441b229e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:37.640013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11c35b-c779-41e7-8a59-28441b229e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:37.653074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:37.715644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11c35b-c779-41e7-8a59-28441b229e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:37.717130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11c35b-c779-41e7-8a59-28441b229e72 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:37.729267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3ea99-94a2-403f-8f50-bfaa7b738ff3 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:37.729547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3ea99-94a2-403f-8f50-bfaa7b738ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:37.729742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3ea99-94a2-403f-8f50-bfaa7b738ff3 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:37.731956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 457/907] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:37.738202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1280aa8-8b7f-4142-81eb-313626c1ab40 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:37.738683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1280aa8-8b7f-4142-81eb-313626c1ab40 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:37.739059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1280aa8-8b7f-4142-81eb-313626c1ab40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:37.762330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-a1280aa8-8b7f-4142-81eb-313626c1ab40 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:37.763676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a1280aa8-8b7f-4142-81eb-313626c1ab40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:37.837088 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 451/908] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:37 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:52:37.842398 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11c35b-c779-41e7-8a59-28441b229e72 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:37.857831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 458/909] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:37.879536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1280aa8-8b7f-4142-81eb-313626c1ab40 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:37.889899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:37.949804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:37.979625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11c35b-c779-41e7-8a59-28441b229e72 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:38.004249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1280aa8-8b7f-4142-81eb-313626c1ab40 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:38.016838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb892d04-baca-43f7-a00a-2ba4afdf55a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:38.017657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb892d04-baca-43f7-a00a-2ba4afdf55a2 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.017971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb892d04-baca-43f7-a00a-2ba4afdf55a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.018207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb892d04-baca-43f7-a00a-2ba4afdf55a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.029202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb892d04-baca-43f7-a00a-2ba4afdf55a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:38.030885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb892d04-baca-43f7-a00a-2ba4afdf55a2 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:38.041988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d1d2a-60d7-4780-a7af-0916cd36ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:38.043063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d1d2a-60d7-4780-a7af-0916cd36ecda None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.043617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d1d2a-60d7-4780-a7af-0916cd36ecda None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.044096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d1d2a-60d7-4780-a7af-0916cd36ecda None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.044975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d1d2a-60d7-4780-a7af-0916cd36ecda None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:38.046541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d1d2a-60d7-4780-a7af-0916cd36ecda None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-fb892d04-baca-43f7-a00a-2ba4afdf55a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:38.049580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:38.050477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:38.051391 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:38.051391 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:38.051391 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:38.051391 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:38.051391 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 452/910] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:38.057342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb892d04-baca-43f7-a00a-2ba4afdf55a2 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-a60d1d2a-60d7-4780-a7af-0916cd36ecda None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:38.062717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:38.063746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 08 01:52:38.064651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:38.067367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 459/911] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:52:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:38.074905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d1d2a-60d7-4780-a7af-0916cd36ecda None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:38.166361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:38.176646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb892d04-baca-43f7-a00a-2ba4afdf55a2 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:38.183023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d1d2a-60d7-4780-a7af-0916cd36ecda None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:38.201407 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:38.216178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80638b9b-9aa5-4954-a4e0-074f38a53534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:38.216847 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80638b9b-9aa5-4954-a4e0-074f38a53534 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.217134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80638b9b-9aa5-4954-a4e0-074f38a53534 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.217410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80638b9b-9aa5-4954-a4e0-074f38a53534 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.218143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80638b9b-9aa5-4954-a4e0-074f38a53534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:38.218494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80638b9b-9aa5-4954-a4e0-074f38a53534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:38.219896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80638b9b-9aa5-4954-a4e0-074f38a53534 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:38.221609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977549d-91ee-42e9-9052-e18cd6311d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:38.222499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977549d-91ee-42e9-9052-e18cd6311d21 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.222954 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977549d-91ee-42e9-9052-e18cd6311d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.223400 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977549d-91ee-42e9-9052-e18cd6311d21 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.224828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977549d-91ee-42e9-9052-e18cd6311d21 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:38.225580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977549d-91ee-42e9-9052-e18cd6311d21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:38.227336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977549d-91ee-42e9-9052-e18cd6311d21 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:38.237567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 453/913] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:38.237858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 460/913] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:52:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:38.239076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80638b9b-9aa5-4954-a4e0-074f38a53534 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:38.239636 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977549d-91ee-42e9-9052-e18cd6311d21 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:38.345958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977549d-91ee-42e9-9052-e18cd6311d21 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:38.353965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80638b9b-9aa5-4954-a4e0-074f38a53534 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:38.381673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a317cf-075e-433a-b6ac-aa52006da037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:38.382471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a317cf-075e-433a-b6ac-aa52006da037 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.382788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a317cf-075e-433a-b6ac-aa52006da037 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.383131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a317cf-075e-433a-b6ac-aa52006da037 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.383902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a317cf-075e-433a-b6ac-aa52006da037 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:38.384360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a317cf-075e-433a-b6ac-aa52006da037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:38.385958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a317cf-075e-433a-b6ac-aa52006da037 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:38.391405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 461/914] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Jun 8 01:52:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:38.394226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a88a43-81df-4023-9cab-3ed652293354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:38.395220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a88a43-81df-4023-9cab-3ed652293354 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.395693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a88a43-81df-4023-9cab-3ed652293354 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.396613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a88a43-81df-4023-9cab-3ed652293354 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.397034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a317cf-075e-433a-b6ac-aa52006da037 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:38.404675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a88a43-81df-4023-9cab-3ed652293354 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:38.406509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a88a43-81df-4023-9cab-3ed652293354 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-14a88a43-81df-4023-9cab-3ed652293354 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:38.422887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:38.423813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:38.424740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:38.424740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:38.424740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:38.424740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:38.424740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:38.424740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:38.424740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:38.424740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:38.427516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 454/915] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:38.436441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a88a43-81df-4023-9cab-3ed652293354 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:38.447606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:38.462095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:38.513520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a317cf-075e-433a-b6ac-aa52006da037 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:38.560477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afafef4-9b86-4e43-b570-6773837d0b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:38.561148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afafef4-9b86-4e43-b570-6773837d0b20 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.561433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afafef4-9b86-4e43-b570-6773837d0b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.561710 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afafef4-9b86-4e43-b570-6773837d0b20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.562452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afafef4-9b86-4e43-b570-6773837d0b20 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:38.563828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afafef4-9b86-4e43-b570-6773837d0b20 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:38.577028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a88a43-81df-4023-9cab-3ed652293354 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:38.597528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 462/916] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jun 8 01:52:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:52:38.633141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28ddf6-c971-4673-98bf-eeab6213d746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:38.634762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28ddf6-c971-4673-98bf-eeab6213d746 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.635611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28ddf6-c971-4673-98bf-eeab6213d746 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.636314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28ddf6-c971-4673-98bf-eeab6213d746 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.637654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28ddf6-c971-4673-98bf-eeab6213d746 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:38.638591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28ddf6-c971-4673-98bf-eeab6213d746 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:38.641035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28ddf6-c971-4673-98bf-eeab6213d746 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:38.661094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 455/917] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:38.670383 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afafef4-9b86-4e43-b570-6773837d0b20 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:38.779098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afafef4-9b86-4e43-b570-6773837d0b20 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:38.780773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:38.818011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d4be4-a3ce-4f80-9450-33fb6a31125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:38.818977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d4be4-a3ce-4f80-9450-33fb6a31125b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.819491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d4be4-a3ce-4f80-9450-33fb6a31125b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.819775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d4be4-a3ce-4f80-9450-33fb6a31125b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.820658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d4be4-a3ce-4f80-9450-33fb6a31125b None admin] RBAC: Adding query filter params (role.id=d3975aa675c345d0974bbdede3f5e506, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=a6aa350eb1504e9c9be9c90870779381) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:38.821009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d4be4-a3ce-4f80-9450-33fb6a31125b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:38.822489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d4be4-a3ce-4f80-9450-33fb6a31125b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:38.829714 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 463/918] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:38 2022] GET /identity/v3/role_assignments?user.id=a6aa350eb1504e9c9be9c90870779381&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=d3975aa675c345d0974bbdede3f5e506 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:38.863484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d7914-576c-49aa-96a2-8217f5c796c0 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.863775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d7914-576c-49aa-96a2-8217f5c796c0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.864059 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d7914-576c-49aa-96a2-8217f5c796c0 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.866019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 456/919] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:38.871796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7e32b-1e37-4376-b657-fcc0a30f8ca5 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:38.872309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7e32b-1e37-4376-b657-fcc0a30f8ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:38.872386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7e32b-1e37-4376-b657-fcc0a30f8ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:38.892459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-beb7e32b-1e37-4376-b657-fcc0a30f8ca5 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:38.893696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-beb7e32b-1e37-4376-b657-fcc0a30f8ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:38.993601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 464/920] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:39.017014 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d7914-576c-49aa-96a2-8217f5c796c0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:39.096480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:39.137235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d7914-576c-49aa-96a2-8217f5c796c0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:39.173590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df05ba2-b0ae-4515-a5fa-244b3d30c1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:39.174422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df05ba2-b0ae-4515-a5fa-244b3d30c1cc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:39.174710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df05ba2-b0ae-4515-a5fa-244b3d30c1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:39.174990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df05ba2-b0ae-4515-a5fa-244b3d30c1cc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:39.175650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df05ba2-b0ae-4515-a5fa-244b3d30c1cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:39.177117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df05ba2-b0ae-4515-a5fa-244b3d30c1cc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-6df05ba2-b0ae-4515-a5fa-244b3d30c1cc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:39.190279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:39.191259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:39.192092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 457/921] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Jun 8 01:52:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:39.198812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7e32b-1e37-4376-b657-fcc0a30f8ca5 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:39.306867 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:39.312903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7e32b-1e37-4376-b657-fcc0a30f8ca5 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:39.322047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c3756-d775-4a98-9343-171b2cfa83bc None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:39.322341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c3756-d775-4a98-9343-171b2cfa83bc None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:39.322620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c3756-d775-4a98-9343-171b2cfa83bc None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:39.324224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 458/922] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:39.329518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1f04b-caf0-44cb-9c69-02803bf17c86 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:39.329811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1f04b-caf0-44cb-9c69-02803bf17c86 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:39.330095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1f04b-caf0-44cb-9c69-02803bf17c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:39.348239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2ba1f04b-caf0-44cb-9c69-02803bf17c86 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:39.348776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2780796-9853-4fd9-b815-7c6c321085fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:39.349475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2ba1f04b-caf0-44cb-9c69-02803bf17c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:39.349771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2780796-9853-4fd9-b815-7c6c321085fd None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:39.350056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2780796-9853-4fd9-b815-7c6c321085fd None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:39.350329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2780796-9853-4fd9-b815-7c6c321085fd None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:39.351029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2780796-9853-4fd9-b815-7c6c321085fd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:39.351331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2780796-9853-4fd9-b815-7c6c321085fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:39.352738 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2780796-9853-4fd9-b815-7c6c321085fd None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:39.358109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 465/923] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:39.364779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2780796-9853-4fd9-b815-7c6c321085fd None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:39.442963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 459/924] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:39.466683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1f04b-caf0-44cb-9c69-02803bf17c86 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:39.482923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2780796-9853-4fd9-b815-7c6c321085fd None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:39.527059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b8577-eb82-4297-abf1-998ad5625cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:39.527996 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b8577-eb82-4297-abf1-998ad5625cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:39.528280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b8577-eb82-4297-abf1-998ad5625cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:39.528582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b8577-eb82-4297-abf1-998ad5625cb3 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:39.529453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b8577-eb82-4297-abf1-998ad5625cb3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:39.529759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b8577-eb82-4297-abf1-998ad5625cb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:39.531685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b8577-eb82-4297-abf1-998ad5625cb3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:39.532118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:39.538646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 466/925] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:39.544745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b8577-eb82-4297-abf1-998ad5625cb3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:39.577647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:39.577959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1f04b-caf0-44cb-9c69-02803bf17c86 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:39.617204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4683b56-0147-490a-86d6-edc250f3650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:39.617942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4683b56-0147-490a-86d6-edc250f3650a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:39.618279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4683b56-0147-490a-86d6-edc250f3650a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:39.618661 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4683b56-0147-490a-86d6-edc250f3650a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:39.619428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4683b56-0147-490a-86d6-edc250f3650a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:39.620857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4683b56-0147-490a-86d6-edc250f3650a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a4683b56-0147-490a-86d6-edc250f3650a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:39.634811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:39.637084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:39.639221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:39.642926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 460/926] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:39.642926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4683b56-0147-490a-86d6-edc250f3650a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:39.666758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b8577-eb82-4297-abf1-998ad5625cb3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:39.710324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd5c51-fb5d-42dd-a154-fe247f5267b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:39.710981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd5c51-fb5d-42dd-a154-fe247f5267b5 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:39.711698 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd5c51-fb5d-42dd-a154-fe247f5267b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:39.711975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd5c51-fb5d-42dd-a154-fe247f5267b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:39.713019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd5c51-fb5d-42dd-a154-fe247f5267b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:39.714573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd5c51-fb5d-42dd-a154-fe247f5267b5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:39.740657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 467/927] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jun 8 01:52:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:39.772284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4683b56-0147-490a-86d6-edc250f3650a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:39.808810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:39.821178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4334f-c50e-419b-8b04-dd82fd92c02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:39.821860 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4334f-c50e-419b-8b04-dd82fd92c02c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:39.822195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4334f-c50e-419b-8b04-dd82fd92c02c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:39.822469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4334f-c50e-419b-8b04-dd82fd92c02c None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:39.823179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4334f-c50e-419b-8b04-dd82fd92c02c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:39.823483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4334f-c50e-419b-8b04-dd82fd92c02c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:39.824920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4334f-c50e-419b-8b04-dd82fd92c02c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:39.830845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 461/928] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jun 8 01:52:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:39.841536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd5c51-fb5d-42dd-a154-fe247f5267b5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:39.940772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:39.969029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd5c51-fb5d-42dd-a154-fe247f5267b5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:40.006752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8749b-cab3-46b6-873d-672c863719ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:40.007778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8749b-cab3-46b6-873d-672c863719ac None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:40.008280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8749b-cab3-46b6-873d-672c863719ac None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:40.008790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8749b-cab3-46b6-873d-672c863719ac None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:40.009729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8749b-cab3-46b6-873d-672c863719ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:40.010235 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8749b-cab3-46b6-873d-672c863719ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:40.011881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8749b-cab3-46b6-873d-672c863719ac None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:40.018639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 468/929] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jun 8 01:52:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:40.025475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4334f-c50e-419b-8b04-dd82fd92c02c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:40.135135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4334f-c50e-419b-8b04-dd82fd92c02c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:40.175716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41aba08-5f4d-4662-84c3-3e701c59d561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:40.176300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aba08-5f4d-4662-84c3-3e701c59d561 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:40.176610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aba08-5f4d-4662-84c3-3e701c59d561 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:40.176886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aba08-5f4d-4662-84c3-3e701c59d561 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:40.178188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41aba08-5f4d-4662-84c3-3e701c59d561 None admin] RBAC: Adding query filter params (interface=public, service_id=b47aee0f2e27490f9bc6512d5ca321cd, region_id=RegionOne) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:40.178499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41aba08-5f4d-4662-84c3-3e701c59d561 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:40.179885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41aba08-5f4d-4662-84c3-3e701c59d561 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:40.185759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 462/930] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jun 8 01:52:40 2022] GET /identity/v3/endpoints?service_id=b47aee0f2e27490f9bc6512d5ca321cd&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:40.192097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8749b-cab3-46b6-873d-672c863719ac None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:40.304107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8749b-cab3-46b6-873d-672c863719ac None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:40.340067 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2651c8e-5bee-4ffe-bb86-68481af9e06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:40.341165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651c8e-5bee-4ffe-bb86-68481af9e06e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:40.341711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651c8e-5bee-4ffe-bb86-68481af9e06e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:40.342217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651c8e-5bee-4ffe-bb86-68481af9e06e None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:40.343105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2651c8e-5bee-4ffe-bb86-68481af9e06e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:40.344734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2651c8e-5bee-4ffe-bb86-68481af9e06e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:40.352236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 469/931] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jun 8 01:52:40 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 01:52:41.071950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e461a-8c90-4856-993a-4b9db100ab98 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:41.072478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e461a-8c90-4856-993a-4b9db100ab98 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:41.072599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e461a-8c90-4856-993a-4b9db100ab98 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:41.074143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 463/932] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:41.079676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f5a37-f59d-4dd8-892c-d35816b9b7e2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:41.080087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f5a37-f59d-4dd8-892c-d35816b9b7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:41.080253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f5a37-f59d-4dd8-892c-d35816b9b7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:41.098637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3d9f5a37-f59d-4dd8-892c-d35816b9b7e2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:41.100121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3d9f5a37-f59d-4dd8-892c-d35816b9b7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:41.102282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:41.171296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 470/933] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:41.182075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce98db-c7ab-4502-ac27-bf9cc515e121 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:41.182365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce98db-c7ab-4502-ac27-bf9cc515e121 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:41.182640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce98db-c7ab-4502-ac27-bf9cc515e121 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:41.184222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 464/934] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:41.189268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e625fd9-067a-4e04-8f70-d3fc92341c13 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:41.189562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e625fd9-067a-4e04-8f70-d3fc92341c13 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:41.189838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e625fd9-067a-4e04-8f70-d3fc92341c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:41.195493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce98db-c7ab-4502-ac27-bf9cc515e121 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:41.210197 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3e625fd9-067a-4e04-8f70-d3fc92341c13 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:41.211388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3e625fd9-067a-4e04-8f70-d3fc92341c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:41.254430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:41.300439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 471/935] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:41.315434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce98db-c7ab-4502-ac27-bf9cc515e121 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:41.322384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e625fd9-067a-4e04-8f70-d3fc92341c13 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:41.356526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:41.357659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:41.358139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:41.358702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:41.359690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:41.361511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:41.363224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: WARNING keystone.server.flask.common [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 08 01:52:41.383510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.sql.core [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 08 01:52:41.383510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 08 01:52:41.383510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [parameters: {'id': '5e81d91bea8f405195fc84399e481bf9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 08 01:52:41.383510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80425) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5e81d91bea8f405195fc84399e481bf9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5e81d91bea8f405195fc84399e481bf9. Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = self._query(query) Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application conn.query(q) Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 08 01:52:41.392034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result.read() Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 08 01:52:41.392948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application next(self.gen) Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application yield resource Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application next(self.gen) Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.session.rollback() Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.force_reraise() Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise self.value Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application session.commit() Jun 08 01:52:41.394125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._prepare_impl() Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.session.flush() Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._flush(objects) Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application compat.raise_( Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application flush_context.execute() Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 08 01:52:41.395038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rec.execute(self) Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 08 01:52:41.396070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = self._query(query) Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application conn.query(q) Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result.read() Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 08 01:52:41.397284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [parameters: {'id': '5e81d91bea8f405195fc84399e481bf9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:41.398183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:41.399045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:41.400165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:41.400165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 08 01:52:41.400165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 08 01:52:41.400165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:41.400165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:41.400165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 08 01:52:41.400165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 08 01:52:41.400165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5e81d91bea8f405195fc84399e481bf9. Jun 08 01:52:41.400165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:41.402225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:41.402609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 465/936] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Jun 8 01:52:41 2022] POST /identity/v3/projects => generated 262 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:41.409037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:41.446112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e625fd9-067a-4e04-8f70-d3fc92341c13 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:41.483819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ca2d0-ee3a-4fd2-86b5-2ca9c246f2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:41.484696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ca2d0-ee3a-4fd2-86b5-2ca9c246f2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:41.484985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ca2d0-ee3a-4fd2-86b5-2ca9c246f2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:41.485263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ca2d0-ee3a-4fd2-86b5-2ca9c246f2b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:41.498085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ca2d0-ee3a-4fd2-86b5-2ca9c246f2b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-b51ca2d0-ee3a-4fd2-86b5-2ca9c246f2b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:41.507081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:41.508040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:52:41.508947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:41.512804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 472/937] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jun 8 01:52:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:41.514560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ca2d0-ee3a-4fd2-86b5-2ca9c246f2b6 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:41.525823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede447a8-8958-44a7-9bb2-0bf9a86e22f1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:41.534717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:41.567997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344a59b-0ce1-4824-ba03-25756cdd5b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:41.569395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a59b-0ce1-4824-ba03-25756cdd5b33 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:41.569901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a59b-0ce1-4824-ba03-25756cdd5b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:41.570369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a59b-0ce1-4824-ba03-25756cdd5b33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:41.579645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344a59b-0ce1-4824-ba03-25756cdd5b33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:41.581913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344a59b-0ce1-4824-ba03-25756cdd5b33 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-b344a59b-0ce1-4824-ba03-25756cdd5b33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:41.602937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:41.603889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:41.604825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:41.604825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:41.604825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:41.604825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:41.604825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:41.604825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:41.604825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:41.604825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:41.604825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 466/938] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:41.610461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344a59b-0ce1-4824-ba03-25756cdd5b33 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:41.643509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ca2d0-ee3a-4fd2-86b5-2ca9c246f2b6 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:41.678270 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:41.687606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e230d-a77e-4958-9a12-4eec606dfc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:41.688282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e230d-a77e-4958-9a12-4eec606dfc6d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:41.688597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e230d-a77e-4958-9a12-4eec606dfc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:41.688875 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e230d-a77e-4958-9a12-4eec606dfc6d None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:41.689624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e230d-a77e-4958-9a12-4eec606dfc6d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:41.690094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e230d-a77e-4958-9a12-4eec606dfc6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:41.691498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e230d-a77e-4958-9a12-4eec606dfc6d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:41.704843 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 473/939] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jun 8 01:52:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:41.706873 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e230d-a77e-4958-9a12-4eec606dfc6d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:41.754641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344a59b-0ce1-4824-ba03-25756cdd5b33 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:41.810092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb1281-8a95-4eb1-8416-155cbe923d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:41.811372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb1281-8a95-4eb1-8416-155cbe923d9e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:41.812288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb1281-8a95-4eb1-8416-155cbe923d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:41.813122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb1281-8a95-4eb1-8416-155cbe923d9e None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:41.814361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb1281-8a95-4eb1-8416-155cbe923d9e None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:41.815278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb1281-8a95-4eb1-8416-155cbe923d9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:41.817302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb1281-8a95-4eb1-8416-155cbe923d9e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:41.818107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e230d-a77e-4958-9a12-4eec606dfc6d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:41.847281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 467/940] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:41.860648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26930ba9-af96-4cae-a3fc-ec8cd0c7b0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:41.861839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26930ba9-af96-4cae-a3fc-ec8cd0c7b0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:41.862361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26930ba9-af96-4cae-a3fc-ec8cd0c7b0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:41.862864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26930ba9-af96-4cae-a3fc-ec8cd0c7b0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:41.876519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26930ba9-af96-4cae-a3fc-ec8cd0c7b0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:41.878459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26930ba9-af96-4cae-a3fc-ec8cd0c7b0d7 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-26930ba9-af96-4cae-a3fc-ec8cd0c7b0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:41.899329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:41.900322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:41.901250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:41.901250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:41.901250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:41.901250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:41.903579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 474/941] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:41.913647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb1281-8a95-4eb1-8416-155cbe923d9e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:41.948097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:42.031149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb1281-8a95-4eb1-8416-155cbe923d9e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:42.069891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f87bc-0f4b-451b-99da-a032793caa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:42.070572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f87bc-0f4b-451b-99da-a032793caa56 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.070856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f87bc-0f4b-451b-99da-a032793caa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.071135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f87bc-0f4b-451b-99da-a032793caa56 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.071862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f87bc-0f4b-451b-99da-a032793caa56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:42.072170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f87bc-0f4b-451b-99da-a032793caa56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:42.073586 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f87bc-0f4b-451b-99da-a032793caa56 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:42.086064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 468/942] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:42.091427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26930ba9-af96-4cae-a3fc-ec8cd0c7b0d7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:42.137567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416be7-a5d7-4846-bc55-4652319e2ba0 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.137895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416be7-a5d7-4846-bc55-4652319e2ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.138206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416be7-a5d7-4846-bc55-4652319e2ba0 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.139792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 469/943] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:42.145441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa52e4c-4471-4224-be76-5d1407a5f54a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.145908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa52e4c-4471-4224-be76-5d1407a5f54a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.146416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa52e4c-4471-4224-be76-5d1407a5f54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.149492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:42.168485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5fa52e4c-4471-4224-be76-5d1407a5f54a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:42.169995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5fa52e4c-4471-4224-be76-5d1407a5f54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:42.221917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26930ba9-af96-4cae-a3fc-ec8cd0c7b0d7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:42.258948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5957a-b810-4a4b-988d-a016a91c1dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:42.259763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5957a-b810-4a4b-988d-a016a91c1dce None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.260040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5957a-b810-4a4b-988d-a016a91c1dce None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.260309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5957a-b810-4a4b-988d-a016a91c1dce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.261530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 470/944] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:42.273546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5957a-b810-4a4b-988d-a016a91c1dce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:42.275397 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5957a-b810-4a4b-988d-a016a91c1dce None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:42.284677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa52e4c-4471-4224-be76-5d1407a5f54a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-10a5957a-b810-4a4b-988d-a016a91c1dce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:42.293290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:42.294276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:42.295232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 475/945] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jun 8 01:52:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:42.300865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5957a-b810-4a4b-988d-a016a91c1dce None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:42.308757 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:42.420759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa52e4c-4471-4224-be76-5d1407a5f54a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:42.422694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5957a-b810-4a4b-988d-a016a91c1dce None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:42.425275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:42.459612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c285b2-3e4b-4fe5-a9fe-eb0df26db2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:42.460809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c285b2-3e4b-4fe5-a9fe-eb0df26db2cc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.461279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c285b2-3e4b-4fe5-a9fe-eb0df26db2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.461734 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c285b2-3e4b-4fe5-a9fe-eb0df26db2cc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.462563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c285b2-3e4b-4fe5-a9fe-eb0df26db2cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:42.464132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c285b2-3e4b-4fe5-a9fe-eb0df26db2cc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:42.468977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566475c2-3b7f-4c5c-9ddb-960c58fb3fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:42.469725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566475c2-3b7f-4c5c-9ddb-960c58fb3fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.470073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566475c2-3b7f-4c5c-9ddb-960c58fb3fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.470416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566475c2-3b7f-4c5c-9ddb-960c58fb3fa5 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.471323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566475c2-3b7f-4c5c-9ddb-960c58fb3fa5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:42.471697 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566475c2-3b7f-4c5c-9ddb-960c58fb3fa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:42.473232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566475c2-3b7f-4c5c-9ddb-960c58fb3fa5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-c5c285b2-3e4b-4fe5-a9fe-eb0df26db2cc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:42.478440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:42.479455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 08 01:52:42.480300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:42.483138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 471/946] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:42.492681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c285b2-3e4b-4fe5-a9fe-eb0df26db2cc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:42.501364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 476/947] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jun 8 01:52:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:42.506395 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566475c2-3b7f-4c5c-9ddb-960c58fb3fa5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:42.605065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:42.619561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566475c2-3b7f-4c5c-9ddb-960c58fb3fa5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:42.631750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c285b2-3e4b-4fe5-a9fe-eb0df26db2cc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:42.659424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea14811-dcf4-4445-88e8-0d83d0d5ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:42.660772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea14811-dcf4-4445-88e8-0d83d0d5ff49 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.661257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea14811-dcf4-4445-88e8-0d83d0d5ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.662525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea14811-dcf4-4445-88e8-0d83d0d5ff49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.674102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea14811-dcf4-4445-88e8-0d83d0d5ff49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:42.675779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea14811-dcf4-4445-88e8-0d83d0d5ff49 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:42.676222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b6ff3-5f06-406f-b889-6fbe695bb5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:42.677186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b6ff3-5f06-406f-b889-6fbe695bb5fc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.677555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b6ff3-5f06-406f-b889-6fbe695bb5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.677944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b6ff3-5f06-406f-b889-6fbe695bb5fc None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.678737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b6ff3-5f06-406f-b889-6fbe695bb5fc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:42.679369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b6ff3-5f06-406f-b889-6fbe695bb5fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:42.681248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b6ff3-5f06-406f-b889-6fbe695bb5fc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:42.692204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 472/948] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:42.694131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b6ff3-5f06-406f-b889-6fbe695bb5fc None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-5ea14811-dcf4-4445-88e8-0d83d0d5ff49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:42.702071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:42.703019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:42.703926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:42.703926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:42.703926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:42.703926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:42.705969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 477/949] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:42.707623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b23b2c-b7b9-4a9c-9e41-5e15670478c9 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.708213 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b23b2c-b7b9-4a9c-9e41-5e15670478c9 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.708855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b23b2c-b7b9-4a9c-9e41-5e15670478c9 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.710418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 478/950] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:42.711482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b23b2c-b7b9-4a9c-9e41-5e15670478c9 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:42.727894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:42.802081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b6ff3-5f06-406f-b889-6fbe695bb5fc None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:42.830031 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b23b2c-b7b9-4a9c-9e41-5e15670478c9 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:42.842429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21eb96-02ec-4347-9a9a-acd1c6d9feb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:42.843106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21eb96-02ec-4347-9a9a-acd1c6d9feb5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.843394 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21eb96-02ec-4347-9a9a-acd1c6d9feb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.843675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21eb96-02ec-4347-9a9a-acd1c6d9feb5 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.844390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21eb96-02ec-4347-9a9a-acd1c6d9feb5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:42.844725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21eb96-02ec-4347-9a9a-acd1c6d9feb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:42.846114 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21eb96-02ec-4347-9a9a-acd1c6d9feb5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:42.851307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 473/951] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:42.853696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b7e2a-da0c-46ed-823f-ddb64f5a504c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.854233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b7e2a-da0c-46ed-823f-ddb64f5a504c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.854916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b7e2a-da0c-46ed-823f-ddb64f5a504c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.867235 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b750e-2840-4fbc-a520-de5f15854763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:42.867836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b750e-2840-4fbc-a520-de5f15854763 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:42.868125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b750e-2840-4fbc-a520-de5f15854763 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:42.868421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b750e-2840-4fbc-a520-de5f15854763 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:42.869188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b750e-2840-4fbc-a520-de5f15854763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:42.869501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b750e-2840-4fbc-a520-de5f15854763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:42.870947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b750e-2840-4fbc-a520-de5f15854763 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:42.879722 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-735b7e2a-da0c-46ed-823f-ddb64f5a504c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:42.882782 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-735b7e2a-da0c-46ed-823f-ddb64f5a504c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:42.885814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:42.888614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 479/952] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:42.890349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b750e-2840-4fbc-a520-de5f15854763 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:42.982957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:43.001905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 474/953] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:43.027661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b750e-2840-4fbc-a520-de5f15854763 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:43.029074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b7e2a-da0c-46ed-823f-ddb64f5a504c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:43.066249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0b3c-b731-4418-915f-b1fa1eb118a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:43.067050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0b3c-b731-4418-915f-b1fa1eb118a9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:43.067339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0b3c-b731-4418-915f-b1fa1eb118a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:43.067619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0b3c-b731-4418-915f-b1fa1eb118a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:43.076380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb0b3c-b731-4418-915f-b1fa1eb118a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:43.077887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb0b3c-b731-4418-915f-b1fa1eb118a9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-effb0b3c-b731-4418-915f-b1fa1eb118a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:43.096286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:43.097744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:43.098722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:43.098722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:43.098722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:43.098722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:43.098722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:43.098722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:43.098722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:43.098722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:43.098722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 480/954] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:43.103416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0b3c-b731-4418-915f-b1fa1eb118a9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:43.149922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b7e2a-da0c-46ed-823f-ddb64f5a504c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:43.189774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae7821-e0c4-4ce4-8672-683d751ad1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:43.191080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae7821-e0c4-4ce4-8672-683d751ad1fe None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:43.191565 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae7821-e0c4-4ce4-8672-683d751ad1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:43.192100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae7821-e0c4-4ce4-8672-683d751ad1fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:43.193308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae7821-e0c4-4ce4-8672-683d751ad1fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:43.194915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae7821-e0c4-4ce4-8672-683d751ad1fe None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d2ae7821-e0c4-4ce4-8672-683d751ad1fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:43.209150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:43.210310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:43.211191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:43.211964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0b3c-b731-4418-915f-b1fa1eb118a9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:43.219153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 475/955] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:43.220436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae7821-e0c4-4ce4-8672-683d751ad1fe None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:43.223609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:43.249852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:43.256748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51840e7-0aea-4dd9-afe1-3ce239ebbf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:43.257327 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51840e7-0aea-4dd9-afe1-3ce239ebbf03 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:43.257999 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51840e7-0aea-4dd9-afe1-3ce239ebbf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:43.258657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51840e7-0aea-4dd9-afe1-3ce239ebbf03 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:43.259824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51840e7-0aea-4dd9-afe1-3ce239ebbf03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:43.260190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51840e7-0aea-4dd9-afe1-3ce239ebbf03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:43.261964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51840e7-0aea-4dd9-afe1-3ce239ebbf03 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:43.279623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 481/956] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:43.280860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51840e7-0aea-4dd9-afe1-3ce239ebbf03 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:43.339548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae7821-e0c4-4ce4-8672-683d751ad1fe None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:43.376312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b69e3-d48c-4d4f-8627-b991a8aa1bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:43.377461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b69e3-d48c-4d4f-8627-b991a8aa1bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:43.377987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b69e3-d48c-4d4f-8627-b991a8aa1bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:43.378461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b69e3-d48c-4d4f-8627-b991a8aa1bd0 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:43.379381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b69e3-d48c-4d4f-8627-b991a8aa1bd0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:43.379911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b69e3-d48c-4d4f-8627-b991a8aa1bd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:43.382523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b69e3-d48c-4d4f-8627-b991a8aa1bd0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:43.393305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51840e7-0aea-4dd9-afe1-3ce239ebbf03 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:43.393818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 476/957] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jun 8 01:52:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:43.394876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b69e3-d48c-4d4f-8627-b991a8aa1bd0 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:43.429559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8942087-4db7-4932-b584-6102228a1486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:43.430522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8942087-4db7-4932-b584-6102228a1486 None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:43.430814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8942087-4db7-4932-b584-6102228a1486 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:43.431093 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8942087-4db7-4932-b584-6102228a1486 None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/a6aa350eb1504e9c9be9c90870779381/roles/d3975aa675c345d0974bbdede3f5e506` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:43.433513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8942087-4db7-4932-b584-6102228a1486 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=a6aa350eb1504e9c9be9c90870779381, role_id=d3975aa675c345d0974bbdede3f5e506)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:43.435146 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8942087-4db7-4932-b584-6102228a1486 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:43.455239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 482/958] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:52:43 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/a6aa350eb1504e9c9be9c90870779381/roles/d3975aa675c345d0974bbdede3f5e506 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 01:52:43.471497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:43.513263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b69e3-d48c-4d4f-8627-b991a8aa1bd0 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:43.551810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d0987-7754-41ba-b500-63e9042d3402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:43.552662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0987-7754-41ba-b500-63e9042d3402 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:43.552947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0987-7754-41ba-b500-63e9042d3402 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:43.553227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0987-7754-41ba-b500-63e9042d3402 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:43.553959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d0987-7754-41ba-b500-63e9042d3402 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:43.554270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d0987-7754-41ba-b500-63e9042d3402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:43.555678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d0987-7754-41ba-b500-63e9042d3402 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:43.561937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 477/959] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jun 8 01:52:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:43.569624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8942087-4db7-4932-b584-6102228a1486 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:43.629992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:43.688925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8942087-4db7-4932-b584-6102228a1486 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:43.725979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a0871-db77-4191-93ae-6347199fb18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:43.726657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a0871-db77-4191-93ae-6347199fb18f None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:43.726943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a0871-db77-4191-93ae-6347199fb18f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:43.727250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a0871-db77-4191-93ae-6347199fb18f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:43.728030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a0871-db77-4191-93ae-6347199fb18f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:43.729626 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a0871-db77-4191-93ae-6347199fb18f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:43.750922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 483/960] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jun 8 01:52:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:44.321371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d13979-97f1-4b7e-954a-dc5bc42b68f3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:44.321888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d13979-97f1-4b7e-954a-dc5bc42b68f3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:44.321966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d13979-97f1-4b7e-954a-dc5bc42b68f3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:44.323415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 478/961] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:44.329487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1783e70-43ad-4e10-b3f4-1afb72ed23d0 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:44.329487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1783e70-43ad-4e10-b3f4-1afb72ed23d0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:44.329487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1783e70-43ad-4e10-b3f4-1afb72ed23d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:44.347754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-a1783e70-43ad-4e10-b3f4-1afb72ed23d0 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:44.349983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a1783e70-43ad-4e10-b3f4-1afb72ed23d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:44.428490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 484/962] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:44.453287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d13979-97f1-4b7e-954a-dc5bc42b68f3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:44.560776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d13979-97f1-4b7e-954a-dc5bc42b68f3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:44.591802 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:44.598023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7de82-cd5a-4d8f-89a8-3e4629581e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:44.598751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7de82-cd5a-4d8f-89a8-3e4629581e86 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:44.599672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7de82-cd5a-4d8f-89a8-3e4629581e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:44.600190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7de82-cd5a-4d8f-89a8-3e4629581e86 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:44.601110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7de82-cd5a-4d8f-89a8-3e4629581e86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:44.602577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7de82-cd5a-4d8f-89a8-3e4629581e86 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:44.616934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.sql.core [None req-f2b7de82-cd5a-4d8f-89a8-3e4629581e86 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 08 01:52:44.616934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 08 01:52:44.616934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [parameters: {'user_id': 'db462b0ac41a486f92511b4e935d4454', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 08 01:52:44.616934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80425) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f2b7de82-cd5a-4d8f-89a8-3e4629581e86 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = self._query(query) Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application conn.query(q) Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result.read() Jun 08 01:52:44.624813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 08 01:52:44.625730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application next(self.gen) Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application yield resource Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application next(self.gen) Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.session.rollback() Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.force_reraise() Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise self.value Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application session.commit() Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 08 01:52:44.626686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._prepare_impl() Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.session.flush() Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._flush(objects) Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application compat.raise_( Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application flush_context.execute() Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rec.execute(self) Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 08 01:52:44.628047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 08 01:52:44.630365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result = self._query(query) Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application conn.query(q) Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application result.read() Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 08 01:52:44.632747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [parameters: {'user_id': 'db462b0ac41a486f92511b4e935d4454', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:44.634719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 08 01:52:44.641830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 08 01:52:44.644352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 08 01:52:44.644352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 08 01:52:44.644352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 08 01:52:44.644352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 08 01:52:44.644352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 08 01:52:44.644352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:44.644352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 479/963] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Jun 8 01:52:44 2022] POST /identity/v3/users => generated 160 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:44.645258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1783e70-43ad-4e10-b3f4-1afb72ed23d0 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:44.684225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:44.759858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1783e70-43ad-4e10-b3f4-1afb72ed23d0 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:44.796564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582608b2-91ad-4744-a8a7-25117bf9c421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:44.797376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582608b2-91ad-4744-a8a7-25117bf9c421 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:44.797663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582608b2-91ad-4744-a8a7-25117bf9c421 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:44.797940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582608b2-91ad-4744-a8a7-25117bf9c421 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:44.806462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582608b2-91ad-4744-a8a7-25117bf9c421 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:44.807988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582608b2-91ad-4744-a8a7-25117bf9c421 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-582608b2-91ad-4744-a8a7-25117bf9c421 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:44.824081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:44.825147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:44.826078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 485/964] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:44.834142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7de82-cd5a-4d8f-89a8-3e4629581e86 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:44.959434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7de82-cd5a-4d8f-89a8-3e4629581e86 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:44.997030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b290f45-6436-48dc-b74a-26ac73c8f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:44.997712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b290f45-6436-48dc-b74a-26ac73c8f67e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:44.998001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b290f45-6436-48dc-b74a-26ac73c8f67e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:44.998274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b290f45-6436-48dc-b74a-26ac73c8f67e None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:44.999081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b290f45-6436-48dc-b74a-26ac73c8f67e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:44.999385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b290f45-6436-48dc-b74a-26ac73c8f67e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:45.001117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b290f45-6436-48dc-b74a-26ac73c8f67e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:45.041921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 480/965] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Jun 8 01:52:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:45.159931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d78aa-1b4e-407e-9844-f4cfa2c94491 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:45.160536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d78aa-1b4e-407e-9844-f4cfa2c94491 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:45.160777 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d78aa-1b4e-407e-9844-f4cfa2c94491 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:45.163080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 486/966] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:45.169909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac44e02-76b3-44e2-9bb3-0bda330d54f8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:45.169909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac44e02-76b3-44e2-9bb3-0bda330d54f8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:45.169909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac44e02-76b3-44e2-9bb3-0bda330d54f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:45.189169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3ac44e02-76b3-44e2-9bb3-0bda330d54f8 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:45.191094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3ac44e02-76b3-44e2-9bb3-0bda330d54f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:45.247142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:45.270207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 481/967] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:45.296732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d78aa-1b4e-407e-9844-f4cfa2c94491 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:45.406569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:45.416022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d78aa-1b4e-407e-9844-f4cfa2c94491 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:45.451263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4810e-97f2-440a-9f32-1f3fc8155707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:45.451852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4810e-97f2-440a-9f32-1f3fc8155707 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:45.452137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4810e-97f2-440a-9f32-1f3fc8155707 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:45.452434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4810e-97f2-440a-9f32-1f3fc8155707 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:45.453169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4810e-97f2-440a-9f32-1f3fc8155707 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:45.454549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4810e-97f2-440a-9f32-1f3fc8155707 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:45.476427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 487/968] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:52:45 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:45.863269 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f102-41bb-4cd8-9e3b-cd8b831f1a93 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:45.863788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f102-41bb-4cd8-9e3b-cd8b831f1a93 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:45.863892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f102-41bb-4cd8-9e3b-cd8b831f1a93 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:45.865994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 482/969] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:45.871072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8d3ac-c435-4573-8903-83b14ada7b57 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:45.871371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8d3ac-c435-4573-8903-83b14ada7b57 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:45.871653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8d3ac-c435-4573-8903-83b14ada7b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:45.888967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-32e8d3ac-c435-4573-8903-83b14ada7b57 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:45.890162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-32e8d3ac-c435-4573-8903-83b14ada7b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:45.973128 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 488/970] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:45.999827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1f102-41bb-4cd8-9e3b-cd8b831f1a93 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.105931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1f102-41bb-4cd8-9e3b-cd8b831f1a93 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:46.139723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77832f79-3fb7-4193-9366-9fc02dc482f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:46.140490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77832f79-3fb7-4193-9366-9fc02dc482f8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.140772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77832f79-3fb7-4193-9366-9fc02dc482f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.141043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77832f79-3fb7-4193-9366-9fc02dc482f8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.151691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc37288-b1d9-419b-aeb0-db88f0f1b0b6 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.151835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77832f79-3fb7-4193-9366-9fc02dc482f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:46.152037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc37288-b1d9-419b-aeb0-db88f0f1b0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.152271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc37288-b1d9-419b-aeb0-db88f0f1b0b6 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.155670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 489/971] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-77832f79-3fb7-4193-9366-9fc02dc482f8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:46.159809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:46.161197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:52:46.163180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:46.164381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10717d39-79ba-4e5b-8cf8-caee307687d4 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.164381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10717d39-79ba-4e5b-8cf8-caee307687d4 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.164381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10717d39-79ba-4e5b-8cf8-caee307687d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.164938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 483/972] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jun 8 01:52:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:46.167568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77832f79-3fb7-4193-9366-9fc02dc482f8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.183100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-10717d39-79ba-4e5b-8cf8-caee307687d4 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:46.184737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-10717d39-79ba-4e5b-8cf8-caee307687d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:46.215807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:46.266674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 490/973] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:46.287200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77832f79-3fb7-4193-9366-9fc02dc482f8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:46.293059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10717d39-79ba-4e5b-8cf8-caee307687d4 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.306951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:46.330858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db5b05-07ff-48ae-b9f7-612eb3e90c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:46.331525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5b05-07ff-48ae-b9f7-612eb3e90c9e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.331817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5b05-07ff-48ae-b9f7-612eb3e90c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.332092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5b05-07ff-48ae-b9f7-612eb3e90c9e None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.332866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db5b05-07ff-48ae-b9f7-612eb3e90c9e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:46.333713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db5b05-07ff-48ae-b9f7-612eb3e90c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:46.335059 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db5b05-07ff-48ae-b9f7-612eb3e90c9e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:46.344528 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 484/974] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jun 8 01:52:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:46.350716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db5b05-07ff-48ae-b9f7-612eb3e90c9e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.405066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10717d39-79ba-4e5b-8cf8-caee307687d4 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:46.441319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d53ad3-3278-4969-86db-b3b8ec2b4aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:46.442396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d53ad3-3278-4969-86db-b3b8ec2b4aac None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.443310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d53ad3-3278-4969-86db-b3b8ec2b4aac None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.443841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d53ad3-3278-4969-86db-b3b8ec2b4aac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.444847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d53ad3-3278-4969-86db-b3b8ec2b4aac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:46.446437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d53ad3-3278-4969-86db-b3b8ec2b4aac None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:46.453237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db5b05-07ff-48ae-b9f7-612eb3e90c9e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:46.454897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-45d53ad3-3278-4969-86db-b3b8ec2b4aac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:46.458798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:46.459903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:46.461263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:46.465082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 491/975] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:46.466417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d53ad3-3278-4969-86db-b3b8ec2b4aac None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.493277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112eb77-dc0d-489b-81e6-5b5f52a762ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:46.494070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112eb77-dc0d-489b-81e6-5b5f52a762ae None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.494354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112eb77-dc0d-489b-81e6-5b5f52a762ae None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.494632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112eb77-dc0d-489b-81e6-5b5f52a762ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.507049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112eb77-dc0d-489b-81e6-5b5f52a762ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:46.508556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112eb77-dc0d-489b-81e6-5b5f52a762ae None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-3112eb77-dc0d-489b-81e6-5b5f52a762ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:46.526417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:46.527395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:46.529251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:46.529251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:46.529251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:46.529251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:46.529251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 485/976] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:46.533239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112eb77-dc0d-489b-81e6-5b5f52a762ae None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.551038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:46.580370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d53ad3-3278-4969-86db-b3b8ec2b4aac None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:46.616621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f90a2-52d0-43a7-8c2e-08bebbfbc7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:46.617318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f90a2-52d0-43a7-8c2e-08bebbfbc7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.617608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f90a2-52d0-43a7-8c2e-08bebbfbc7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.617888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f90a2-52d0-43a7-8c2e-08bebbfbc7d5 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.618619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f90a2-52d0-43a7-8c2e-08bebbfbc7d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:46.618933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f90a2-52d0-43a7-8c2e-08bebbfbc7d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:46.620378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f90a2-52d0-43a7-8c2e-08bebbfbc7d5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:46.631116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 492/977] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jun 8 01:52:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:46.632316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f90a2-52d0-43a7-8c2e-08bebbfbc7d5 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.647122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112eb77-dc0d-489b-81e6-5b5f52a762ae None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:46.688603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610a9cc-9280-492f-8eeb-aefd7b8a246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:46.689199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610a9cc-9280-492f-8eeb-aefd7b8a246d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.690002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610a9cc-9280-492f-8eeb-aefd7b8a246d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.690298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610a9cc-9280-492f-8eeb-aefd7b8a246d None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.691294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610a9cc-9280-492f-8eeb-aefd7b8a246d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:46.691610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610a9cc-9280-492f-8eeb-aefd7b8a246d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:46.693725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610a9cc-9280-492f-8eeb-aefd7b8a246d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:46.706958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 486/978] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:46.712956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610a9cc-9280-492f-8eeb-aefd7b8a246d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.728552 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:46.749159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f90a2-52d0-43a7-8c2e-08bebbfbc7d5 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:46.785718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504796-2af2-4fc8-a38f-f6e6ee657748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:46.786585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504796-2af2-4fc8-a38f-f6e6ee657748 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.786900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504796-2af2-4fc8-a38f-f6e6ee657748 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.787180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504796-2af2-4fc8-a38f-f6e6ee657748 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.787893 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8504796-2af2-4fc8-a38f-f6e6ee657748 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:46.788201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8504796-2af2-4fc8-a38f-f6e6ee657748 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:46.789633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8504796-2af2-4fc8-a38f-f6e6ee657748 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:46.794754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 493/979] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jun 8 01:52:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:46.800582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504796-2af2-4fc8-a38f-f6e6ee657748 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.803773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:46.830092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610a9cc-9280-492f-8eeb-aefd7b8a246d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:46.867305 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92b87b-b18b-44fd-ba5e-eb1a5d1eeabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:46.868124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92b87b-b18b-44fd-ba5e-eb1a5d1eeabd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.868423 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92b87b-b18b-44fd-ba5e-eb1a5d1eeabd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.868710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92b87b-b18b-44fd-ba5e-eb1a5d1eeabd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.876136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92b87b-b18b-44fd-ba5e-eb1a5d1eeabd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:46.877608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92b87b-b18b-44fd-ba5e-eb1a5d1eeabd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-ad92b87b-b18b-44fd-ba5e-eb1a5d1eeabd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:46.892471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:46.893482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:46.894472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 487/980] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jun 8 01:52:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:46.899997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92b87b-b18b-44fd-ba5e-eb1a5d1eeabd None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.914905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504796-2af2-4fc8-a38f-f6e6ee657748 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:46.951640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef97d4-215e-4026-b585-7c2efd6c9fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:46.952283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef97d4-215e-4026-b585-7c2efd6c9fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:46.952704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef97d4-215e-4026-b585-7c2efd6c9fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:46.952985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef97d4-215e-4026-b585-7c2efd6c9fc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:46.953910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef97d4-215e-4026-b585-7c2efd6c9fc9 None admin] RBAC: Adding query filter params (interface=admin, service_id=b47aee0f2e27490f9bc6512d5ca321cd, region_id=RegionOne) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:46.954227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef97d4-215e-4026-b585-7c2efd6c9fc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:46.955612 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef97d4-215e-4026-b585-7c2efd6c9fc9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:46.966275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 494/981] 10.222.0.20 () {56 vars in 1249 bytes} [Wed Jun 8 01:52:46 2022] GET /identity/v3/endpoints?service_id=b47aee0f2e27490f9bc6512d5ca321cd&interface=admin®ion_id=RegionOne => generated 193 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:46.967106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef97d4-215e-4026-b585-7c2efd6c9fc9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:46.985455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:47.014620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92b87b-b18b-44fd-ba5e-eb1a5d1eeabd None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:47.042243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:47.051445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207043b-26f2-42db-8e16-dd666d63c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:47.052710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207043b-26f2-42db-8e16-dd666d63c09e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:47.053466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207043b-26f2-42db-8e16-dd666d63c09e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:47.053466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207043b-26f2-42db-8e16-dd666d63c09e None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:47.054091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207043b-26f2-42db-8e16-dd666d63c09e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:47.054402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207043b-26f2-42db-8e16-dd666d63c09e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:47.055852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207043b-26f2-42db-8e16-dd666d63c09e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:47.074151 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 488/982] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Jun 8 01:52:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:47.079695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207043b-26f2-42db-8e16-dd666d63c09e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:47.086838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef97d4-215e-4026-b585-7c2efd6c9fc9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:47.125906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107063ad-d8d9-436e-81ec-955a4c079898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:47.126585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107063ad-d8d9-436e-81ec-955a4c079898 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:47.126875 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107063ad-d8d9-436e-81ec-955a4c079898 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:47.127156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107063ad-d8d9-436e-81ec-955a4c079898 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:47.127842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107063ad-d8d9-436e-81ec-955a4c079898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:47.129247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107063ad-d8d9-436e-81ec-955a4c079898 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:47.144424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 495/983] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jun 8 01:52:46 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 01:52:47.196725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207043b-26f2-42db-8e16-dd666d63c09e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:47.230263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:47.236353 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13128296-5a03-43b5-8a28-f28ae4688712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:47.237051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13128296-5a03-43b5-8a28-f28ae4688712 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:47.237336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13128296-5a03-43b5-8a28-f28ae4688712 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:47.237613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13128296-5a03-43b5-8a28-f28ae4688712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:47.313053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13128296-5a03-43b5-8a28-f28ae4688712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:47.314550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13128296-5a03-43b5-8a28-f28ae4688712 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:47.433568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 489/984] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:47 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 08 01:52:47.439385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107063ad-d8d9-436e-81ec-955a4c079898 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:47.545476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107063ad-d8d9-436e-81ec-955a4c079898 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:47.574452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:47.581426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8261c1b-440f-498b-bdc8-de710b3caa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:47.582224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8261c1b-440f-498b-bdc8-de710b3caa69 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:47.582507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8261c1b-440f-498b-bdc8-de710b3caa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:47.582782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8261c1b-440f-498b-bdc8-de710b3caa69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:47.594731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8261c1b-440f-498b-bdc8-de710b3caa69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:47.596183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8261c1b-440f-498b-bdc8-de710b3caa69 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-e8261c1b-440f-498b-bdc8-de710b3caa69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:47.613794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:47.615479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:47.616495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:47.616495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:47.616495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:47.616495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:47.619652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 496/985] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:47.621071 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13128296-5a03-43b5-8a28-f28ae4688712 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:47.652782 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:47.731057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13128296-5a03-43b5-8a28-f28ae4688712 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:47.764608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e074f-916a-4fa7-a89a-6f66b4d4bfd1 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:47.764898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e074f-916a-4fa7-a89a-6f66b4d4bfd1 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:47.765177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e074f-916a-4fa7-a89a-6f66b4d4bfd1 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:47.766244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e30d1-acdd-4d67-92f8-3cffa852d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:47.766713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 497/986] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:47.766969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e30d1-acdd-4d67-92f8-3cffa852d02f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:47.767255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e30d1-acdd-4d67-92f8-3cffa852d02f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:47.767595 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e30d1-acdd-4d67-92f8-3cffa852d02f None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:47.768307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e30d1-acdd-4d67-92f8-3cffa852d02f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:47.768644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e30d1-acdd-4d67-92f8-3cffa852d02f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:47.770240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e30d1-acdd-4d67-92f8-3cffa852d02f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:47.771353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641eb44-8e31-492e-a855-5d2a4cc61eef None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:47.771831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641eb44-8e31-492e-a855-5d2a4cc61eef None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:47.772113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641eb44-8e31-492e-a855-5d2a4cc61eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:47.786934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 490/987] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:47.788111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e30d1-acdd-4d67-92f8-3cffa852d02f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:47.789377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-6641eb44-8e31-492e-a855-5d2a4cc61eef None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:47.790582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6641eb44-8e31-492e-a855-5d2a4cc61eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:47.869782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 498/988] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:47.891873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:47.892370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641eb44-8e31-492e-a855-5d2a4cc61eef None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:47.901564 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e30d1-acdd-4d67-92f8-3cffa852d02f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:47.940078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edf99d-c36c-4b89-a77e-25a7113229e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:47.940922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edf99d-c36c-4b89-a77e-25a7113229e1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:47.941208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edf99d-c36c-4b89-a77e-25a7113229e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:47.941489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edf99d-c36c-4b89-a77e-25a7113229e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:47.948815 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edf99d-c36c-4b89-a77e-25a7113229e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:47.950267 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edf99d-c36c-4b89-a77e-25a7113229e1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-50edf99d-c36c-4b89-a77e-25a7113229e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:47.964922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:47.965850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:47.967555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:47.967555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:47.967555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:47.967555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:47.967555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:47.967555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:47.967555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:47.967555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:47.967555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 491/989] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:47.971722 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edf99d-c36c-4b89-a77e-25a7113229e1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:48.014887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641eb44-8e31-492e-a855-5d2a4cc61eef None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:48.030839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:48.056754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7a64a-06f9-4563-b1e1-a7c595536637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:48.057610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7a64a-06f9-4563-b1e1-a7c595536637 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:48.057902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7a64a-06f9-4563-b1e1-a7c595536637 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:48.058183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7a64a-06f9-4563-b1e1-a7c595536637 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:48.058844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7a64a-06f9-4563-b1e1-a7c595536637 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:48.060222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7a64a-06f9-4563-b1e1-a7c595536637 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-90a7a64a-06f9-4563-b1e1-a7c595536637 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:48.072427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:48.073439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:48.074288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 499/990] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:48.079357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7a64a-06f9-4563-b1e1-a7c595536637 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:48.082476 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edf99d-c36c-4b89-a77e-25a7113229e1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:48.118740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46cf2f-6100-42c6-a73b-0a48be609c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:48.119407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46cf2f-6100-42c6-a73b-0a48be609c5c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:48.119693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46cf2f-6100-42c6-a73b-0a48be609c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:48.120022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46cf2f-6100-42c6-a73b-0a48be609c5c None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:48.120785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46cf2f-6100-42c6-a73b-0a48be609c5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:48.121101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46cf2f-6100-42c6-a73b-0a48be609c5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:48.122542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46cf2f-6100-42c6-a73b-0a48be609c5c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:48.134835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 492/991] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:48.140336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46cf2f-6100-42c6-a73b-0a48be609c5c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:48.154894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:48.185951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7a64a-06f9-4563-b1e1-a7c595536637 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:48.219374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067274b-cccd-4034-9d8b-af9c23f5998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:48.220039 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067274b-cccd-4034-9d8b-af9c23f5998e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:48.220321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067274b-cccd-4034-9d8b-af9c23f5998e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:48.220624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067274b-cccd-4034-9d8b-af9c23f5998e None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:48.221320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067274b-cccd-4034-9d8b-af9c23f5998e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:48.221627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067274b-cccd-4034-9d8b-af9c23f5998e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:48.222998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067274b-cccd-4034-9d8b-af9c23f5998e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:48.227959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 500/992] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:48.233410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067274b-cccd-4034-9d8b-af9c23f5998e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:48.256051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46cf2f-6100-42c6-a73b-0a48be609c5c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:48.275663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:48.295696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6088e2-23ea-4efe-99d5-f6ba57057349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:48.296598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6088e2-23ea-4efe-99d5-f6ba57057349 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:48.296997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6088e2-23ea-4efe-99d5-f6ba57057349 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:48.297375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6088e2-23ea-4efe-99d5-f6ba57057349 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:48.298452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6088e2-23ea-4efe-99d5-f6ba57057349 None admin] RBAC: Adding query filter params (role.id=d3975aa675c345d0974bbdede3f5e506, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=a6aa350eb1504e9c9be9c90870779381) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:48.299041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6088e2-23ea-4efe-99d5-f6ba57057349 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:48.300653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6088e2-23ea-4efe-99d5-f6ba57057349 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:48.320430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 493/993] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:48 2022] GET /identity/v3/role_assignments?user.id=a6aa350eb1504e9c9be9c90870779381&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=d3975aa675c345d0974bbdede3f5e506 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:48.349806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067274b-cccd-4034-9d8b-af9c23f5998e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:48.391402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb887c-20d0-4d17-9857-279bb5add50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:48.392104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb887c-20d0-4d17-9857-279bb5add50d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:48.392484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb887c-20d0-4d17-9857-279bb5add50d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:48.392829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb887c-20d0-4d17-9857-279bb5add50d None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:48.393628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb887c-20d0-4d17-9857-279bb5add50d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:48.394023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb887c-20d0-4d17-9857-279bb5add50d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:48.396180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb887c-20d0-4d17-9857-279bb5add50d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:48.408838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 501/994] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:48.409040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6088e2-23ea-4efe-99d5-f6ba57057349 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:48.499491 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:48.526791 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6088e2-23ea-4efe-99d5-f6ba57057349 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:48.562211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b0e2d-f240-4eac-ad91-11a8b6580a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:48.562883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b0e2d-f240-4eac-ad91-11a8b6580a39 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:48.563163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b0e2d-f240-4eac-ad91-11a8b6580a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:48.563435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b0e2d-f240-4eac-ad91-11a8b6580a39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:48.564221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b0e2d-f240-4eac-ad91-11a8b6580a39 None admin] RBAC: Adding query filter params (interface=public, service_id=bb1bc44810a946ab9be7028b92e15aa7, region_id=RegionOne) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:48.564562 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b0e2d-f240-4eac-ad91-11a8b6580a39 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:48.565949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b0e2d-f240-4eac-ad91-11a8b6580a39 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:48.572093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 494/995] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jun 8 01:52:48 2022] GET /identity/v3/endpoints?service_id=bb1bc44810a946ab9be7028b92e15aa7&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:48.577857 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb887c-20d0-4d17-9857-279bb5add50d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:48.682140 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb887c-20d0-4d17-9857-279bb5add50d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:48.684421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:48.719389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d74c9-45e6-4eff-9cd5-c1f55fa44596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:48.720066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d74c9-45e6-4eff-9cd5-c1f55fa44596 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:48.720352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d74c9-45e6-4eff-9cd5-c1f55fa44596 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:48.720661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d74c9-45e6-4eff-9cd5-c1f55fa44596 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:48.721353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d74c9-45e6-4eff-9cd5-c1f55fa44596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:48.722899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d74c9-45e6-4eff-9cd5-c1f55fa44596 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:48.732156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 502/996] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jun 8 01:52:48 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 01:52:49.472177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaf93e-06fc-4d73-b33d-56b388988852 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:49.472738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaf93e-06fc-4d73-b33d-56b388988852 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:49.472849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaf93e-06fc-4d73-b33d-56b388988852 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:49.474276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 495/997] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:49.480555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aff6cd-fa06-4365-993d-c0d5bd6ab63e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:49.480863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aff6cd-fa06-4365-993d-c0d5bd6ab63e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:49.481134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aff6cd-fa06-4365-993d-c0d5bd6ab63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:49.498094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-43aff6cd-fa06-4365-993d-c0d5bd6ab63e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:49.499284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-43aff6cd-fa06-4365-993d-c0d5bd6ab63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:49.570757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 503/998] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:49.593074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaf93e-06fc-4d73-b33d-56b388988852 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:49.696118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eaf93e-06fc-4d73-b33d-56b388988852 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:49.728248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7c8c9-c493-48c0-afaf-d7d7226f6b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:49.729012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7c8c9-c493-48c0-afaf-d7d7226f6b3f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:49.729296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7c8c9-c493-48c0-afaf-d7d7226f6b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:49.729571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7c8c9-c493-48c0-afaf-d7d7226f6b3f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:49.730210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7c8c9-c493-48c0-afaf-d7d7226f6b3f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:49.731573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7c8c9-c493-48c0-afaf-d7d7226f6b3f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-40c7c8c9-c493-48c0-afaf-d7d7226f6b3f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:49.743798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:49.744808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 08 01:52:49.745658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:49.747180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 496/999] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Jun 8 01:52:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:49.750990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aff6cd-fa06-4365-993d-c0d5bd6ab63e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:49.847961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:49.852747 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aff6cd-fa06-4365-993d-c0d5bd6ab63e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:49.891473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dacfa-5fb1-4eab-a67b-768ae4409b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:49.892144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dacfa-5fb1-4eab-a67b-768ae4409b35 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:49.892454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dacfa-5fb1-4eab-a67b-768ae4409b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:49.892759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9dacfa-5fb1-4eab-a67b-768ae4409b35 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:49.893469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dacfa-5fb1-4eab-a67b-768ae4409b35 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:49.893774 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dacfa-5fb1-4eab-a67b-768ae4409b35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:49.895145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9dacfa-5fb1-4eab-a67b-768ae4409b35 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:49.905426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 504/1000] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jun 8 01:52:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:49.906683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7c8c9-c493-48c0-afaf-d7d7226f6b3f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:49.912631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:50.018278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7c8c9-c493-48c0-afaf-d7d7226f6b3f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:50.054377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec21ff-4349-40e8-9e79-a3d6de8a3558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:50.055055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec21ff-4349-40e8-9e79-a3d6de8a3558 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:50.055339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec21ff-4349-40e8-9e79-a3d6de8a3558 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:50.055614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec21ff-4349-40e8-9e79-a3d6de8a3558 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:50.056338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec21ff-4349-40e8-9e79-a3d6de8a3558 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:50.056654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec21ff-4349-40e8-9e79-a3d6de8a3558 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:50.058026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec21ff-4349-40e8-9e79-a3d6de8a3558 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:50.063227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 497/1001] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Jun 8 01:52:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:50.068725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dacfa-5fb1-4eab-a67b-768ae4409b35 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:50.170036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9dacfa-5fb1-4eab-a67b-768ae4409b35 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:50.213086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec27457-a854-4d64-a158-4050916a53e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:50.213986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec27457-a854-4d64-a158-4050916a53e3 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:50.214276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec27457-a854-4d64-a158-4050916a53e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:50.214559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec27457-a854-4d64-a158-4050916a53e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:50.215322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec27457-a854-4d64-a158-4050916a53e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:50.216804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec27457-a854-4d64-a158-4050916a53e3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:50.234223 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 505/1002] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jun 8 01:52:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:50.639653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef8198-f562-4d5e-897e-60d5f97e3d07 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:50.640209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef8198-f562-4d5e-897e-60d5f97e3d07 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:50.640209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef8198-f562-4d5e-897e-60d5f97e3d07 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:50.641809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 498/1003] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:50.647531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7c931-67f8-4baa-a1ea-922e0ea6168b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:50.647830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7c931-67f8-4baa-a1ea-922e0ea6168b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:50.648114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7c931-67f8-4baa-a1ea-922e0ea6168b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:50.665237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-5ea7c931-67f8-4baa-a1ea-922e0ea6168b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:50.666435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5ea7c931-67f8-4baa-a1ea-922e0ea6168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:50.684642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:50.750532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 506/1004] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:50.773633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef8198-f562-4d5e-897e-60d5f97e3d07 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:50.842378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:50.879253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef8198-f562-4d5e-897e-60d5f97e3d07 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:50.922724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ddc18-54a3-4aef-a97d-e794acaa3832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:50.923537 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ddc18-54a3-4aef-a97d-e794acaa3832 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:50.923827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ddc18-54a3-4aef-a97d-e794acaa3832 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:50.924102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ddc18-54a3-4aef-a97d-e794acaa3832 None admin] PATH_INFO: `/v3/services/image` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:50.924790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ddc18-54a3-4aef-a97d-e794acaa3832 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:50.926201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ddc18-54a3-4aef-a97d-e794acaa3832 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a85ddc18-54a3-4aef-a97d-e794acaa3832 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:50.938213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:50.939219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 08 01:52:50.940107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:50.941741 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 499/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Jun 8 01:52:50 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:50.946085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7c931-67f8-4baa-a1ea-922e0ea6168b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:51.031768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa092a5-f5f4-47d7-8b15-b484565964e8 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:51.032051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa092a5-f5f4-47d7-8b15-b484565964e8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:51.032320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa092a5-f5f4-47d7-8b15-b484565964e8 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:51.033740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 500/1006] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:51.039019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76736fe1-1b4d-4d21-893c-8d4d5efa799a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:51.039306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76736fe1-1b4d-4d21-893c-8d4d5efa799a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:51.039581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76736fe1-1b4d-4d21-893c-8d4d5efa799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:51.053953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7c931-67f8-4baa-a1ea-922e0ea6168b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:51.056439 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-76736fe1-1b4d-4d21-893c-8d4d5efa799a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:51.057608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-76736fe1-1b4d-4d21-893c-8d4d5efa799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:51.091101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f149b-8a18-47af-a205-c6d6bef48489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:51.091772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f149b-8a18-47af-a205-c6d6bef48489 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:51.092054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f149b-8a18-47af-a205-c6d6bef48489 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:51.092330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f149b-8a18-47af-a205-c6d6bef48489 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:51.093075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f149b-8a18-47af-a205-c6d6bef48489 None admin] RBAC: Adding query filter params (name=image) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:51.093384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f149b-8a18-47af-a205-c6d6bef48489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:51.094753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f149b-8a18-47af-a205-c6d6bef48489 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:51.104352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 507/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:51.105384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f149b-8a18-47af-a205-c6d6bef48489 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:51.121033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:51.138366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 501/1008] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:51.171061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76736fe1-1b4d-4d21-893c-8d4d5efa799a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:51.210240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f149b-8a18-47af-a205-c6d6bef48489 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:51.231668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:51.246698 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e966b9a-78fd-4d4b-bcf7-969cdd044baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:51.247759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e966b9a-78fd-4d4b-bcf7-969cdd044baa None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:51.248042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e966b9a-78fd-4d4b-bcf7-969cdd044baa None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:51.248317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e966b9a-78fd-4d4b-bcf7-969cdd044baa None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:51.249395 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e966b9a-78fd-4d4b-bcf7-969cdd044baa None admin] RBAC: Adding query filter params (type=image) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:51.249708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e966b9a-78fd-4d4b-bcf7-969cdd044baa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:51.251379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e966b9a-78fd-4d4b-bcf7-969cdd044baa None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:51.264442 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 508/1009] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:51.283346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76736fe1-1b4d-4d21-893c-8d4d5efa799a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:51.322378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decf172-7697-4201-a74f-7edf633f3ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:51.323210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decf172-7697-4201-a74f-7edf633f3ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:51.323495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decf172-7697-4201-a74f-7edf633f3ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:51.323774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6decf172-7697-4201-a74f-7edf633f3ee6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:51.324472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6decf172-7697-4201-a74f-7edf633f3ee6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:51.326016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6decf172-7697-4201-a74f-7edf633f3ee6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-6decf172-7697-4201-a74f-7edf633f3ee6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:51.339613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:51.340759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 08 01:52:51.341686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:51.346121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 502/1010] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:51.347383 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e966b9a-78fd-4d4b-bcf7-969cdd044baa None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:51.438692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:51.469078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e966b9a-78fd-4d4b-bcf7-969cdd044baa None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:51.507979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bdcb1-bea6-4f6a-ae57-acc4b99df81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:51.508690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bdcb1-bea6-4f6a-ae57-acc4b99df81d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:51.508979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bdcb1-bea6-4f6a-ae57-acc4b99df81d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:51.509259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bdcb1-bea6-4f6a-ae57-acc4b99df81d None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:51.509975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bdcb1-bea6-4f6a-ae57-acc4b99df81d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:51.510286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bdcb1-bea6-4f6a-ae57-acc4b99df81d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:51.511672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bdcb1-bea6-4f6a-ae57-acc4b99df81d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:51.520957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 509/1011] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:51.523256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decf172-7697-4201-a74f-7edf633f3ee6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:51.628170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6decf172-7697-4201-a74f-7edf633f3ee6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:51.655141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:51.666563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776c220-7e12-4ed2-be7c-efc8761098a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:51.667240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776c220-7e12-4ed2-be7c-efc8761098a1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:51.667525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776c220-7e12-4ed2-be7c-efc8761098a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:51.667804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776c220-7e12-4ed2-be7c-efc8761098a1 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:51.668543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776c220-7e12-4ed2-be7c-efc8761098a1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:51.668855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776c220-7e12-4ed2-be7c-efc8761098a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:51.670227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776c220-7e12-4ed2-be7c-efc8761098a1 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:51.680546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 503/1012] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Jun 8 01:52:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:51.682044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bdcb1-bea6-4f6a-ae57-acc4b99df81d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:51.789118 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bdcb1-bea6-4f6a-ae57-acc4b99df81d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:51.824840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c5c0a-56fd-4543-8fbc-b6cae47b8914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:51.825502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5c0a-56fd-4543-8fbc-b6cae47b8914 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:51.825787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5c0a-56fd-4543-8fbc-b6cae47b8914 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:51.826063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c5c0a-56fd-4543-8fbc-b6cae47b8914 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:51.826811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c5c0a-56fd-4543-8fbc-b6cae47b8914 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:51.828180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c5c0a-56fd-4543-8fbc-b6cae47b8914 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:51.836894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:51.856427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 510/1013] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jun 8 01:52:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:52.578397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c124e9-35fe-4adc-9771-8f9a49673288 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:52.578732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c124e9-35fe-4adc-9771-8f9a49673288 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:52.578947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c124e9-35fe-4adc-9771-8f9a49673288 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:52.580461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 504/1014] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:52.586212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e564f-621c-4be2-8f24-57b1e522a895 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:52.586651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e564f-621c-4be2-8f24-57b1e522a895 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:52.586813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e564f-621c-4be2-8f24-57b1e522a895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:52.604850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7a5e564f-621c-4be2-8f24-57b1e522a895 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:52.606027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7a5e564f-621c-4be2-8f24-57b1e522a895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:52.684975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 511/1015] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:52 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:52.709707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c124e9-35fe-4adc-9771-8f9a49673288 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:52.815744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c124e9-35fe-4adc-9771-8f9a49673288 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:52.851108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42172a43-8243-48aa-9918-f1f0440d8194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:52.851824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42172a43-8243-48aa-9918-f1f0440d8194 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:52.852102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42172a43-8243-48aa-9918-f1f0440d8194 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:52.852370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42172a43-8243-48aa-9918-f1f0440d8194 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:52.853553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42172a43-8243-48aa-9918-f1f0440d8194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:52.854976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42172a43-8243-48aa-9918-f1f0440d8194 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:52.861316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 505/1016] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:52.862980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e564f-621c-4be2-8f24-57b1e522a895 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:52.971078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e564f-621c-4be2-8f24-57b1e522a895 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:52.995115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:53.007940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df65b7-aff8-48e2-bd4e-14d5e53b90af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:53.008642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df65b7-aff8-48e2-bd4e-14d5e53b90af None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:53.008928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df65b7-aff8-48e2-bd4e-14d5e53b90af None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:53.009201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df65b7-aff8-48e2-bd4e-14d5e53b90af None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:53.009906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df65b7-aff8-48e2-bd4e-14d5e53b90af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:53.011347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df65b7-aff8-48e2-bd4e-14d5e53b90af None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:53.041176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 512/1017] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:52:52 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:52:53.543950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ddc2e-536e-4d8b-8662-5b114eb74804 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:53.544329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ddc2e-536e-4d8b-8662-5b114eb74804 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:53.544556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ddc2e-536e-4d8b-8662-5b114eb74804 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:53.546086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 506/1018] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:53.551205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ddf7f-1208-4034-86b8-e6f476a1c8ab None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:53.552218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ddf7f-1208-4034-86b8-e6f476a1c8ab None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:53.552743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ddf7f-1208-4034-86b8-e6f476a1c8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:53.570373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-685ddf7f-1208-4034-86b8-e6f476a1c8ab None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:53.571584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-685ddf7f-1208-4034-86b8-e6f476a1c8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:53.640573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 513/1019] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:53 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:53.664671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ddc2e-536e-4d8b-8662-5b114eb74804 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:53.733802 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:53.778596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ddc2e-536e-4d8b-8662-5b114eb74804 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:53.813160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a22fe-efcb-4473-b79d-079e30550ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:53.813754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a22fe-efcb-4473-b79d-079e30550ebc None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:53.814035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a22fe-efcb-4473-b79d-079e30550ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:53.814307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a22fe-efcb-4473-b79d-079e30550ebc None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:53.815033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a22fe-efcb-4473-b79d-079e30550ebc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:53.816420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a22fe-efcb-4473-b79d-079e30550ebc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:53.832436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 507/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:52:53 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:52:54.232524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454c6a8-ba4f-4107-94bc-7a2fb6255b35 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:54.232721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454c6a8-ba4f-4107-94bc-7a2fb6255b35 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:54.232991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454c6a8-ba4f-4107-94bc-7a2fb6255b35 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:54.234516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 514/1021] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:54.240036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f90f78-f726-4365-8be8-f66d0751d685 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:54.240330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f90f78-f726-4365-8be8-f66d0751d685 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:54.240720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f90f78-f726-4365-8be8-f66d0751d685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:54.257268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-76f90f78-f726-4365-8be8-f66d0751d685 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:54.258445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-76f90f78-f726-4365-8be8-f66d0751d685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:54.336884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 508/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:54.361137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454c6a8-ba4f-4107-94bc-7a2fb6255b35 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:54.465941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454c6a8-ba4f-4107-94bc-7a2fb6255b35 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:54.468265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:54.503061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d33389-5a83-4f0d-a820-d532458ca4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:54.503875 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d33389-5a83-4f0d-a820-d532458ca4ee None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:54.504157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d33389-5a83-4f0d-a820-d532458ca4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:54.504454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d33389-5a83-4f0d-a820-d532458ca4ee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:54.515020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d33389-5a83-4f0d-a820-d532458ca4ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-61d33389-5a83-4f0d-a820-d532458ca4ee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:54.523719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:54.524879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:54.530795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 515/1023] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jun 8 01:52:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:54.531482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f90f78-f726-4365-8be8-f66d0751d685 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:54.578192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:54.637366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f90f78-f726-4365-8be8-f66d0751d685 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:54.674038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b1716-2940-4be2-bf6b-f0d0541ef4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:54.674717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b1716-2940-4be2-bf6b-f0d0541ef4ec None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:54.675000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b1716-2940-4be2-bf6b-f0d0541ef4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:54.675278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b1716-2940-4be2-bf6b-f0d0541ef4ec None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:54.676031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b1716-2940-4be2-bf6b-f0d0541ef4ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:54.676338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b1716-2940-4be2-bf6b-f0d0541ef4ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:54.678608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b1716-2940-4be2-bf6b-f0d0541ef4ec None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:54.691714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 509/1024] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jun 8 01:52:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:54.693844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d33389-5a83-4f0d-a820-d532458ca4ee None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:54.794379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d33389-5a83-4f0d-a820-d532458ca4ee None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:54.827919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540194d-fb73-4c71-8198-d71dab5433a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:54.828735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540194d-fb73-4c71-8198-d71dab5433a9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:54.829021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540194d-fb73-4c71-8198-d71dab5433a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:54.829298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540194d-fb73-4c71-8198-d71dab5433a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:54.839876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540194d-fb73-4c71-8198-d71dab5433a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:54.841365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540194d-fb73-4c71-8198-d71dab5433a9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-e540194d-fb73-4c71-8198-d71dab5433a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:54.858718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:54.859637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:54.865598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:54.865598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:54.865598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:54.865598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:54.865598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 516/1025] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:54.865889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b1716-2940-4be2-bf6b-f0d0541ef4ec None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:54.973265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b1716-2940-4be2-bf6b-f0d0541ef4ec None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:54.981567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:55.011399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad965d47-e9b6-4c7e-a9a5-cd17b07f438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:55.012064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad965d47-e9b6-4c7e-a9a5-cd17b07f438e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:55.012345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad965d47-e9b6-4c7e-a9a5-cd17b07f438e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:55.012644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad965d47-e9b6-4c7e-a9a5-cd17b07f438e None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:55.013361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad965d47-e9b6-4c7e-a9a5-cd17b07f438e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:55.013664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad965d47-e9b6-4c7e-a9a5-cd17b07f438e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:55.015044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad965d47-e9b6-4c7e-a9a5-cd17b07f438e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:55.032347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 510/1026] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:55.033678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540194d-fb73-4c71-8198-d71dab5433a9 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:55.055330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:55.139775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540194d-fb73-4c71-8198-d71dab5433a9 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:55.173215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fa64b-9c57-4100-9abc-1610b096f17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:55.174011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fa64b-9c57-4100-9abc-1610b096f17f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:55.174292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fa64b-9c57-4100-9abc-1610b096f17f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:55.174563 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fa64b-9c57-4100-9abc-1610b096f17f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:55.182559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fa64b-9c57-4100-9abc-1610b096f17f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:55.184004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fa64b-9c57-4100-9abc-1610b096f17f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-ac1fa64b-9c57-4100-9abc-1610b096f17f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:55.198028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:55.198988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:55.204827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 517/1027] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jun 8 01:52:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:55.205434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad965d47-e9b6-4c7e-a9a5-cd17b07f438e None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:55.304655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad965d47-e9b6-4c7e-a9a5-cd17b07f438e None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:55.338896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eaa909-bc46-4754-bc4c-9e2e372e6d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:55.339489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eaa909-bc46-4754-bc4c-9e2e372e6d31 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:55.339768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eaa909-bc46-4754-bc4c-9e2e372e6d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:55.340040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eaa909-bc46-4754-bc4c-9e2e372e6d31 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:55.340890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eaa909-bc46-4754-bc4c-9e2e372e6d31 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:55.341199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eaa909-bc46-4754-bc4c-9e2e372e6d31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:55.342617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eaa909-bc46-4754-bc4c-9e2e372e6d31 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:55.364325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 511/1028] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jun 8 01:52:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:55.365585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fa64b-9c57-4100-9abc-1610b096f17f None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:55.377047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc2cf-747c-466d-a9a1-110d3e226fcb None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:55.377338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc2cf-747c-466d-a9a1-110d3e226fcb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:55.377614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc2cf-747c-466d-a9a1-110d3e226fcb None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:55.383074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 512/1029] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:55.384453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd33a5-9db3-4ce5-b6a6-093816d34005 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:55.384738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd33a5-9db3-4ce5-b6a6-093816d34005 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:55.385012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd33a5-9db3-4ce5-b6a6-093816d34005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:55.403443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-60cd33a5-9db3-4ce5-b6a6-093816d34005 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:55.404654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-60cd33a5-9db3-4ce5-b6a6-093816d34005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:55.422777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:55.449258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:55.474913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fa64b-9c57-4100-9abc-1610b096f17f None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:55.478604 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 513/1030] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:55.502868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd33a5-9db3-4ce5-b6a6-093816d34005 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:55.511068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519301a7-0232-4e51-b511-5f8f32cfde12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:55.511726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519301a7-0232-4e51-b511-5f8f32cfde12 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:55.512009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519301a7-0232-4e51-b511-5f8f32cfde12 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:55.512284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519301a7-0232-4e51-b511-5f8f32cfde12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:55.589017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519301a7-0232-4e51-b511-5f8f32cfde12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:55.590865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519301a7-0232-4e51-b511-5f8f32cfde12 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:55.609799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd33a5-9db3-4ce5-b6a6-093816d34005 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:55.646995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ac88-8864-485b-ad4a-bfec588cb56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:55.647797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ac88-8864-485b-ad4a-bfec588cb56a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:55.648086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ac88-8864-485b-ad4a-bfec588cb56a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:55.648363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ac88-8864-485b-ad4a-bfec588cb56a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:55.649589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855ac88-8864-485b-ad4a-bfec588cb56a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:55.651040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855ac88-8864-485b-ad4a-bfec588cb56a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:55.653044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 514/1031] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:55.658300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ac88-8864-485b-ad4a-bfec588cb56a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:55.678669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:55.685107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:55.706876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 518/1032] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:52:55.713809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519301a7-0232-4e51-b511-5f8f32cfde12 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:55.778343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ac88-8864-485b-ad4a-bfec588cb56a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:55.824183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc350943-20e6-4d42-a468-2327635c28c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:55.824888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc350943-20e6-4d42-a468-2327635c28c5 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:55.825169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc350943-20e6-4d42-a468-2327635c28c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:55.825442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc350943-20e6-4d42-a468-2327635c28c5 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:55.825769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519301a7-0232-4e51-b511-5f8f32cfde12 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:55.826144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc350943-20e6-4d42-a468-2327635c28c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:55.827563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc350943-20e6-4d42-a468-2327635c28c5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:55.849711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 515/1033] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:52:55 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:52:55.877857 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e1050-f614-4ca4-b656-07d27acc2aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:55.878684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e1050-f614-4ca4-b656-07d27acc2aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:55.878971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e1050-f614-4ca4-b656-07d27acc2aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:55.879247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e1050-f614-4ca4-b656-07d27acc2aa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:55.896560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e1050-f614-4ca4-b656-07d27acc2aa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:55.898086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e1050-f614-4ca4-b656-07d27acc2aa4 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-299e1050-f614-4ca4-b656-07d27acc2aa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:55.924015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:55.925027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:55.925952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:55.925952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:55.925952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:55.925952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:55.925952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 519/1034] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:55.931841 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc350943-20e6-4d42-a468-2327635c28c5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:56.019406 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:56.047555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc350943-20e6-4d42-a468-2327635c28c5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:56.088219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e4b6c-0f53-4ba5-8285-bcc7ec00fbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:56.088922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4b6c-0f53-4ba5-8285-bcc7ec00fbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:56.089199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4b6c-0f53-4ba5-8285-bcc7ec00fbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:56.089478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e4b6c-0f53-4ba5-8285-bcc7ec00fbe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:56.090197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e4b6c-0f53-4ba5-8285-bcc7ec00fbe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:56.090503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e4b6c-0f53-4ba5-8285-bcc7ec00fbe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:56.091900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e4b6c-0f53-4ba5-8285-bcc7ec00fbe0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:56.105867 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 516/1035] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:56.111304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e1050-f614-4ca4-b656-07d27acc2aa4 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:56.137534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41231e6-1f81-4718-ae2f-544f02ea3f4a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:56.138110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41231e6-1f81-4718-ae2f-544f02ea3f4a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:56.138566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41231e6-1f81-4718-ae2f-544f02ea3f4a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:56.140434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 517/1036] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:56.144359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:56.145536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f062d-73f6-4666-b80b-bd490afed75c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:56.145834 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f062d-73f6-4666-b80b-bd490afed75c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:56.146103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f062d-73f6-4666-b80b-bd490afed75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:56.166218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5f7f062d-73f6-4666-b80b-bd490afed75c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:56.167402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5f7f062d-73f6-4666-b80b-bd490afed75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:56.227310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e1050-f614-4ca4-b656-07d27acc2aa4 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:56.243607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 518/1037] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:56.265282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf8dc3-4d71-4b7a-a51e-012d6c1ff26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:56.266101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf8dc3-4d71-4b7a-a51e-012d6c1ff26a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:56.266335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f062d-73f6-4666-b80b-bd490afed75c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:56.266457 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf8dc3-4d71-4b7a-a51e-012d6c1ff26a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:56.266638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf8dc3-4d71-4b7a-a51e-012d6c1ff26a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:56.274784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf8dc3-4d71-4b7a-a51e-012d6c1ff26a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:56.276373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf8dc3-4d71-4b7a-a51e-012d6c1ff26a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-e9bf8dc3-4d71-4b7a-a51e-012d6c1ff26a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:52:56.292819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:56.293868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:56.294926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:52:56.294926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:52:56.294926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:52:56.294926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:52:56.294926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:52:56.294926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:52:56.294926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:52:56.294926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:56.294926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 520/1038] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:52:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:56.299903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf8dc3-4d71-4b7a-a51e-012d6c1ff26a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:56.344646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:56.382771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f062d-73f6-4666-b80b-bd490afed75c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:56.396656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:56.411716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf8dc3-4d71-4b7a-a51e-012d6c1ff26a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:56.420825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda95a32-17f0-4d65-9b02-718ff935001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:56.421968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda95a32-17f0-4d65-9b02-718ff935001c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:56.422248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda95a32-17f0-4d65-9b02-718ff935001c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:56.422831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda95a32-17f0-4d65-9b02-718ff935001c None admin] PATH_INFO: `/v3/services/image` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:56.423487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda95a32-17f0-4d65-9b02-718ff935001c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:56.424881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda95a32-17f0-4d65-9b02-718ff935001c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-fda95a32-17f0-4d65-9b02-718ff935001c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:56.436805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:56.437891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 08 01:52:56.438766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:56.444037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 519/1039] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Jun 8 01:52:56 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:56.444934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda95a32-17f0-4d65-9b02-718ff935001c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:56.449587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6e9af-940b-45b5-b72d-87655c85bffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:56.450253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6e9af-940b-45b5-b72d-87655c85bffb None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:56.450535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6e9af-940b-45b5-b72d-87655c85bffb None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:56.450810 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6e9af-940b-45b5-b72d-87655c85bffb None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:56.451596 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6e9af-940b-45b5-b72d-87655c85bffb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:56.451961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6e9af-940b-45b5-b72d-87655c85bffb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:56.453431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6e9af-940b-45b5-b72d-87655c85bffb None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:56.471536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 521/1040] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:52:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:56.472318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6e9af-940b-45b5-b72d-87655c85bffb None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:56.548835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda95a32-17f0-4d65-9b02-718ff935001c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:56.580679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:56.582811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6e9af-940b-45b5-b72d-87655c85bffb None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:56.587654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573f102-5875-420f-83e8-788701614c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:56.588322 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573f102-5875-420f-83e8-788701614c36 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:56.588635 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573f102-5875-420f-83e8-788701614c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:56.588911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573f102-5875-420f-83e8-788701614c36 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:56.589603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573f102-5875-420f-83e8-788701614c36 None admin] RBAC: Adding query filter params (name=image) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:56.589899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573f102-5875-420f-83e8-788701614c36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:56.591263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573f102-5875-420f-83e8-788701614c36 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:56.601166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 520/1041] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:56.602189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573f102-5875-420f-83e8-788701614c36 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:56.624266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81ca03-d175-41ff-82e9-3e5ac9c431a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:56.624980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81ca03-d175-41ff-82e9-3e5ac9c431a6 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:56.625263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81ca03-d175-41ff-82e9-3e5ac9c431a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:56.625818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81ca03-d175-41ff-82e9-3e5ac9c431a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:56.626368 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81ca03-d175-41ff-82e9-3e5ac9c431a6 None admin] RBAC: Adding query filter params (role.id=d3975aa675c345d0974bbdede3f5e506, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:56.626685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81ca03-d175-41ff-82e9-3e5ac9c431a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:56.628218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81ca03-d175-41ff-82e9-3e5ac9c431a6 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:56.634752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 522/1042] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:56 2022] GET /identity/v3/role_assignments?user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=d3975aa675c345d0974bbdede3f5e506 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:56.717247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573f102-5875-420f-83e8-788701614c36 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:56.754322 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593d84b-2d10-44cb-84e6-fbfebf7553a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:56.755186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593d84b-2d10-44cb-84e6-fbfebf7553a4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:56.755521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593d84b-2d10-44cb-84e6-fbfebf7553a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:56.755981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593d84b-2d10-44cb-84e6-fbfebf7553a4 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:56.756604 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593d84b-2d10-44cb-84e6-fbfebf7553a4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:56.756918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593d84b-2d10-44cb-84e6-fbfebf7553a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:56.758356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593d84b-2d10-44cb-84e6-fbfebf7553a4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:56.768861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 521/1043] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:56.770056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81ca03-d175-41ff-82e9-3e5ac9c431a6 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:56.788559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:56.882166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81ca03-d175-41ff-82e9-3e5ac9c431a6 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:56.923033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b7102-ae1b-48c2-9618-7abf4af43d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:56.923697 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b7102-ae1b-48c2-9618-7abf4af43d7b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:56.923978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b7102-ae1b-48c2-9618-7abf4af43d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:56.924253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b7102-ae1b-48c2-9618-7abf4af43d7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:56.925069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b7102-ae1b-48c2-9618-7abf4af43d7b None admin] RBAC: Adding query filter params (interface=public, service_id=56b406952e1f45bea8354b4357e2936c, region_id=RegionOne) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:56.925379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b7102-ae1b-48c2-9618-7abf4af43d7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:56.926861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b7102-ae1b-48c2-9618-7abf4af43d7b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:56.936668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 523/1044] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jun 8 01:52:56 2022] GET /identity/v3/endpoints?service_id=56b406952e1f45bea8354b4357e2936c&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:56.938144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593d84b-2d10-44cb-84e6-fbfebf7553a4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:57.000337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:57.044701 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593d84b-2d10-44cb-84e6-fbfebf7553a4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:57.079274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852dea9-3971-45a0-8a93-294e8b577bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:57.079966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852dea9-3971-45a0-8a93-294e8b577bde None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:57.080245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852dea9-3971-45a0-8a93-294e8b577bde None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:57.080560 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852dea9-3971-45a0-8a93-294e8b577bde None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:57.081288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852dea9-3971-45a0-8a93-294e8b577bde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:57.082720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852dea9-3971-45a0-8a93-294e8b577bde None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:57.096432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 522/1045] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jun 8 01:52:56 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 01:52:58.227627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce409-fc2f-4555-bdfa-d44a4fe8361e None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:58.228121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce409-fc2f-4555-bdfa-d44a4fe8361e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:58.228121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce409-fc2f-4555-bdfa-d44a4fe8361e None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:58.229622 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 524/1046] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:58.234692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719270a-f4b0-440f-9e16-c4330256313f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:58.235072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719270a-f4b0-440f-9e16-c4330256313f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:58.235256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719270a-f4b0-440f-9e16-c4330256313f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:58.253613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7719270a-f4b0-440f-9e16-c4330256313f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:58.254870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7719270a-f4b0-440f-9e16-c4330256313f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:58.326675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 523/1047] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:58.350239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ce409-fc2f-4555-bdfa-d44a4fe8361e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:58.430531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:58.456883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ce409-fc2f-4555-bdfa-d44a4fe8361e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:58.492247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd091ae-7c32-4b44-873f-731c5ae34d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:58.493001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd091ae-7c32-4b44-873f-731c5ae34d5c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:58.493288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd091ae-7c32-4b44-873f-731c5ae34d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:58.493562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd091ae-7c32-4b44-873f-731c5ae34d5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:58.503842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd091ae-7c32-4b44-873f-731c5ae34d5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-edd091ae-7c32-4b44-873f-731c5ae34d5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:58.511936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:58.512906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:58.518700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 525/1048] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:52:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:52:58.519426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719270a-f4b0-440f-9e16-c4330256313f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:58.580197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:58.625649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719270a-f4b0-440f-9e16-c4330256313f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:58.660563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf972ce-339c-4858-9076-3cd7c6bd6b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:58.661875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf972ce-339c-4858-9076-3cd7c6bd6b4f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:58.662158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf972ce-339c-4858-9076-3cd7c6bd6b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:58.662434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf972ce-339c-4858-9076-3cd7c6bd6b4f None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:58.663173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf972ce-339c-4858-9076-3cd7c6bd6b4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:58.663479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf972ce-339c-4858-9076-3cd7c6bd6b4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:58.664900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf972ce-339c-4858-9076-3cd7c6bd6b4f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:58.678295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 524/1049] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:52:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:58.679540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd091ae-7c32-4b44-873f-731c5ae34d5c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:58.787170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd091ae-7c32-4b44-873f-731c5ae34d5c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:58.821481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10db85a-5772-477c-976e-72e469c1d777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:58.822362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10db85a-5772-477c-976e-72e469c1d777 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:58.822647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10db85a-5772-477c-976e-72e469c1d777 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:58.822949 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10db85a-5772-477c-976e-72e469c1d777 None admin] PATH_INFO: `/v3/users/df1e72a7024f43fe837ade3cbf393411` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:58.825719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:58.838527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10db85a-5772-477c-976e-72e469c1d777 None admin] RBAC: Authorizing `identity:get_user(user_id=df1e72a7024f43fe837ade3cbf393411)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:58.839986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10db85a-5772-477c-976e-72e469c1d777 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:58.846370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 526/1050] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:52:58 2022] GET /identity/v3/users/df1e72a7024f43fe837ade3cbf393411 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:58.847456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf972ce-339c-4858-9076-3cd7c6bd6b4f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:58.874993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41894a-420f-4bf1-8fd9-7508540662fd None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:58.875329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41894a-420f-4bf1-8fd9-7508540662fd None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:58.875550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41894a-420f-4bf1-8fd9-7508540662fd None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:58.876929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 527/1051] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:52:58.881834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d31ce-e83d-4592-ae61-7ef4a4d99384 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:58.882129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d31ce-e83d-4592-ae61-7ef4a4d99384 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:58.882415 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d31ce-e83d-4592-ae61-7ef4a4d99384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:58.899836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e64d31ce-e83d-4592-ae61-7ef4a4d99384 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:58.901071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e64d31ce-e83d-4592-ae61-7ef4a4d99384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:58.953711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf972ce-339c-4858-9076-3cd7c6bd6b4f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:58.970491 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:58.976666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 528/1052] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:52:58.992137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04a5a4-8f56-4074-a538-daa49f446b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:58.992833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04a5a4-8f56-4074-a538-daa49f446b54 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:58.993117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04a5a4-8f56-4074-a538-daa49f446b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:58.993397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04a5a4-8f56-4074-a538-daa49f446b54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:58.999065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d31ce-e83d-4592-ae61-7ef4a4d99384 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.067732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04a5a4-8f56-4074-a538-daa49f446b54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:59.069103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04a5a4-8f56-4074-a538-daa49f446b54 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:59.104849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d31ce-e83d-4592-ae61-7ef4a4d99384 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:59.129027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:59.142342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c51ffd-70d4-4715-9782-351ded051bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:59.143188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c51ffd-70d4-4715-9782-351ded051bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.143474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c51ffd-70d4-4715-9782-351ded051bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.143749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c51ffd-70d4-4715-9782-351ded051bd6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.154942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c51ffd-70d4-4715-9782-351ded051bd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-79c51ffd-70d4-4715-9782-351ded051bd6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:52:59.163956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:59.165194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:59.166280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 529/1053] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jun 8 01:52:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:59.170881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c51ffd-70d4-4715-9782-351ded051bd6 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.177030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 525/1054] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:52:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:52:59.182715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04a5a4-8f56-4074-a538-daa49f446b54 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.200244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:59.287074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04a5a4-8f56-4074-a538-daa49f446b54 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:59.292406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c51ffd-70d4-4715-9782-351ded051bd6 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:59.322848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb3256-af1e-46bd-9cac-c3c3b993b322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:59.323744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb3256-af1e-46bd-9cac-c3c3b993b322 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.324316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb3256-af1e-46bd-9cac-c3c3b993b322 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.324472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb3256-af1e-46bd-9cac-c3c3b993b322 None admin] PATH_INFO: `/v3/projects/a8579efc8c884a66b99f031edb4065ca` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.325613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb3256-af1e-46bd-9cac-c3c3b993b322 None admin] RBAC: Authorizing `identity:get_project(project_id=a8579efc8c884a66b99f031edb4065ca)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:59.327005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb3256-af1e-46bd-9cac-c3c3b993b322 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:59.328944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129cc65-a9c9-4c2f-9cd7-41739af0f4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:59.329664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129cc65-a9c9-4c2f-9cd7-41739af0f4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.329903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129cc65-a9c9-4c2f-9cd7-41739af0f4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.330182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129cc65-a9c9-4c2f-9cd7-41739af0f4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.330983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129cc65-a9c9-4c2f-9cd7-41739af0f4d2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:59.331292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129cc65-a9c9-4c2f-9cd7-41739af0f4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:59.332843 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129cc65-a9c9-4c2f-9cd7-41739af0f4d2 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:59.334208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 526/1055] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:52:59 2022] GET /identity/v3/projects/a8579efc8c884a66b99f031edb4065ca => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:59.335111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb3256-af1e-46bd-9cac-c3c3b993b322 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.341287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 530/1056] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jun 8 01:52:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:59.347068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129cc65-a9c9-4c2f-9cd7-41739af0f4d2 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.395767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:59.424652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:59.440887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb3256-af1e-46bd-9cac-c3c3b993b322 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:59.449189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129cc65-a9c9-4c2f-9cd7-41739af0f4d2 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:59.475287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638331c-0cdb-4474-8127-a42941edee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:59.475939 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638331c-0cdb-4474-8127-a42941edee37 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.476224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638331c-0cdb-4474-8127-a42941edee37 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.476523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638331c-0cdb-4474-8127-a42941edee37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.477336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638331c-0cdb-4474-8127-a42941edee37 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=a8579efc8c884a66b99f031edb4065ca, user.id=df1e72a7024f43fe837ade3cbf393411) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:59.477645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638331c-0cdb-4474-8127-a42941edee37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:59.479082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638331c-0cdb-4474-8127-a42941edee37 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:59.482430 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b6318-4240-4a3a-9f68-e4dbd6b1b246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:59.483210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b6318-4240-4a3a-9f68-e4dbd6b1b246 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.483491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b6318-4240-4a3a-9f68-e4dbd6b1b246 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.483768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b6318-4240-4a3a-9f68-e4dbd6b1b246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.484376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 527/1057] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:52:59 2022] GET /identity/v3/role_assignments?user.id=df1e72a7024f43fe837ade3cbf393411&scope.project.id=a8579efc8c884a66b99f031edb4065ca&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:52:59.485600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73b3c-cc31-47d8-9345-d769e9fae0a3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.485891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73b3c-cc31-47d8-9345-d769e9fae0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.486160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73b3c-cc31-47d8-9345-d769e9fae0a3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.487404 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 528/1058] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:52:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:52:59.498285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4265d1-9253-4494-b888-bd0f9a264a0d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.498644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4265d1-9253-4494-b888-bd0f9a264a0d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.498833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4265d1-9253-4494-b888-bd0f9a264a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.502169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b6318-4240-4a3a-9f68-e4dbd6b1b246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:59.503684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b6318-4240-4a3a-9f68-e4dbd6b1b246 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:59.519328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-eb4265d1-9253-4494-b888-bd0f9a264a0d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:52:59.520905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-eb4265d1-9253-4494-b888-bd0f9a264a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-a67b6318-4240-4a3a-9f68-e4dbd6b1b246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:52:59.523801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:59.524800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:59.525886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:52:59.525886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:52:59.525886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:52:59.525886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:59.525886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 531/1059] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:59.530970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b6318-4240-4a3a-9f68-e4dbd6b1b246 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.600084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 529/1060] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:52:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:52:59.623851 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4265d1-9253-4494-b888-bd0f9a264a0d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.642594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:59.649037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b6318-4240-4a3a-9f68-e4dbd6b1b246 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:59.697662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8092e91-43aa-4a24-af2e-7fcfcceba413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:59.698509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8092e91-43aa-4a24-af2e-7fcfcceba413 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.698798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8092e91-43aa-4a24-af2e-7fcfcceba413 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.699074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8092e91-43aa-4a24-af2e-7fcfcceba413 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.699806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8092e91-43aa-4a24-af2e-7fcfcceba413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:59.700115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8092e91-43aa-4a24-af2e-7fcfcceba413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:59.701582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8092e91-43aa-4a24-af2e-7fcfcceba413 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:59.714251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:59.714586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 532/1061] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:59.719925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8092e91-43aa-4a24-af2e-7fcfcceba413 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.739516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4265d1-9253-4494-b888-bd0f9a264a0d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:59.775303 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b932-0db1-432b-82f5-3819d81a9f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:59.776098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b932-0db1-432b-82f5-3819d81a9f0b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.776382 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b932-0db1-432b-82f5-3819d81a9f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.776684 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b932-0db1-432b-82f5-3819d81a9f0b None admin] PATH_INFO: `/v3/services/image` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.777331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8b932-0db1-432b-82f5-3819d81a9f0b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:59.778723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8b932-0db1-432b-82f5-3819d81a9f0b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-26d8b932-0db1-432b-82f5-3819d81a9f0b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:59.791002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:59.792055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:52:59.792960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 530/1062] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Jun 8 01:52:59 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:59.798276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b932-0db1-432b-82f5-3819d81a9f0b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.831144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8092e91-43aa-4a24-af2e-7fcfcceba413 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:59.868013 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e48dd-770b-48e8-b260-9f8873a0ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:59.868765 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e48dd-770b-48e8-b260-9f8873a0ce1b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.869049 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e48dd-770b-48e8-b260-9f8873a0ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.869322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e48dd-770b-48e8-b260-9f8873a0ce1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.877087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e48dd-770b-48e8-b260-9f8873a0ce1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:59.878692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e48dd-770b-48e8-b260-9f8873a0ce1b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-f88e48dd-770b-48e8-b260-9f8873a0ce1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:52:59.896726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:52:59.897742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:52:59.898716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 533/1063] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jun 8 01:52:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:52:59.903636 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e48dd-770b-48e8-b260-9f8873a0ce1b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.904999 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b932-0db1-432b-82f5-3819d81a9f0b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:52:59.929193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:52:59.951092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f9e00-5942-4d1f-bce9-bb901bc3a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:52:59.951765 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f9e00-5942-4d1f-bce9-bb901bc3a871 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:52:59.952046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f9e00-5942-4d1f-bce9-bb901bc3a871 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:52:59.952319 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f9e00-5942-4d1f-bce9-bb901bc3a871 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:52:59.953054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f9e00-5942-4d1f-bce9-bb901bc3a871 None admin] RBAC: Adding query filter params (name=image) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:52:59.953359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f9e00-5942-4d1f-bce9-bb901bc3a871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:52:59.954728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f9e00-5942-4d1f-bce9-bb901bc3a871 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:52:59.960115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 531/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:52:59.965678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f9e00-5942-4d1f-bce9-bb901bc3a871 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:52:59.976058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:00.020888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e48dd-770b-48e8-b260-9f8873a0ce1b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:00.056940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f02e91-cf20-4891-8313-4c78d4dde735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:00.057616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f02e91-cf20-4891-8313-4c78d4dde735 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:00.057899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f02e91-cf20-4891-8313-4c78d4dde735 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:00.058173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f02e91-cf20-4891-8313-4c78d4dde735 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:00.059007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f02e91-cf20-4891-8313-4c78d4dde735 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:00.059324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f02e91-cf20-4891-8313-4c78d4dde735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:00.060768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f02e91-cf20-4891-8313-4c78d4dde735 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:00.077065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f9e00-5942-4d1f-bce9-bb901bc3a871 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:00.078847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 534/1065] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jun 8 01:52:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:00.083987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f02e91-cf20-4891-8313-4c78d4dde735 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:00.112962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576cc41-d060-40fe-8ddc-75d0b415fe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:00.113935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576cc41-d060-40fe-8ddc-75d0b415fe36 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:00.114577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576cc41-d060-40fe-8ddc-75d0b415fe36 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:00.114864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576cc41-d060-40fe-8ddc-75d0b415fe36 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:00.115568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576cc41-d060-40fe-8ddc-75d0b415fe36 None admin] RBAC: Adding query filter params (type=image) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:00.115880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576cc41-d060-40fe-8ddc-75d0b415fe36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:00.117275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576cc41-d060-40fe-8ddc-75d0b415fe36 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:00.127476 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 532/1066] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:52:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:00.128670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576cc41-d060-40fe-8ddc-75d0b415fe36 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:00.167239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:00.194020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f02e91-cf20-4891-8313-4c78d4dde735 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:00.199937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:00.231651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576cc41-d060-40fe-8ddc-75d0b415fe36 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:00.232739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320642f-a46b-4744-9d32-d22a47c7fb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:00.233658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320642f-a46b-4744-9d32-d22a47c7fb53 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:00.234056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320642f-a46b-4744-9d32-d22a47c7fb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:00.234345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320642f-a46b-4744-9d32-d22a47c7fb53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:00.245640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320642f-a46b-4744-9d32-d22a47c7fb53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:00.247311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320642f-a46b-4744-9d32-d22a47c7fb53 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-7320642f-a46b-4744-9d32-d22a47c7fb53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:53:00.267181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:00.268161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:00.270468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:53:00.270468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:53:00.270468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:53:00.270468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:53:00.270468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 535/1067] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:53:00.271732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135c55-05ac-48cb-8f1d-9240a3260959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:00.272418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135c55-05ac-48cb-8f1d-9240a3260959 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:00.272716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135c55-05ac-48cb-8f1d-9240a3260959 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:00.272987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135c55-05ac-48cb-8f1d-9240a3260959 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:00.273993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55135c55-05ac-48cb-8f1d-9240a3260959 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:00.274397 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320642f-a46b-4744-9d32-d22a47c7fb53 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:00.275356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55135c55-05ac-48cb-8f1d-9240a3260959 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:00.298963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 533/1068] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jun 8 01:53:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:53:00.388699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320642f-a46b-4744-9d32-d22a47c7fb53 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:00.420703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:00.436927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd97b79-8f09-4a52-9508-eb48d3bf2aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:00.437606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd97b79-8f09-4a52-9508-eb48d3bf2aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:00.437887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd97b79-8f09-4a52-9508-eb48d3bf2aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:00.438155 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd97b79-8f09-4a52-9508-eb48d3bf2aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:00.438871 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd97b79-8f09-4a52-9508-eb48d3bf2aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:00.439175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd97b79-8f09-4a52-9508-eb48d3bf2aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:00.440590 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd97b79-8f09-4a52-9508-eb48d3bf2aa0 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:00.453513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 536/1069] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:00.460262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135c55-05ac-48cb-8f1d-9240a3260959 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:00.569127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135c55-05ac-48cb-8f1d-9240a3260959 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:00.594134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:00.605431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d89ac-819b-42bd-9fce-416e8781ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:00.606256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d89ac-819b-42bd-9fce-416e8781ad73 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:00.606540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d89ac-819b-42bd-9fce-416e8781ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:00.606809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d89ac-819b-42bd-9fce-416e8781ad73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:00.614744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d89ac-819b-42bd-9fce-416e8781ad73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:00.616249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d89ac-819b-42bd-9fce-416e8781ad73 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2f1d89ac-819b-42bd-9fce-416e8781ad73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:53:00.632597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:00.633570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:00.635091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:53:00.635091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:53:00.635091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:53:00.635091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:53:00.635091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:53:00.635091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:53:00.635091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:53:00.635091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:53:00.635091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 534/1070] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:53:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:00.640636 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd97b79-8f09-4a52-9508-eb48d3bf2aa0 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:00.749589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd97b79-8f09-4a52-9508-eb48d3bf2aa0 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:00.784030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997999d3-8e99-48e4-92d5-f3fda17d11aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:00.784971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997999d3-8e99-48e4-92d5-f3fda17d11aa None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:00.785306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997999d3-8e99-48e4-92d5-f3fda17d11aa None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:00.786053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997999d3-8e99-48e4-92d5-f3fda17d11aa None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:00.786853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997999d3-8e99-48e4-92d5-f3fda17d11aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:00.787166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997999d3-8e99-48e4-92d5-f3fda17d11aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:00.788630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997999d3-8e99-48e4-92d5-f3fda17d11aa None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:00.800881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 537/1071] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:53:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:00.807068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d89ac-819b-42bd-9fce-416e8781ad73 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:00.916663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d89ac-819b-42bd-9fce-416e8781ad73 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:00.950592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3833e7-b4a9-456b-8bda-2120ff028a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:00.951624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3833e7-b4a9-456b-8bda-2120ff028a80 None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:00.952025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3833e7-b4a9-456b-8bda-2120ff028a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:00.952413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3833e7-b4a9-456b-8bda-2120ff028a80 None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/0ef930dc7b7a4fb29f7b32e464e9eb3b/roles/d3975aa675c345d0974bbdede3f5e506` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:00.954695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3833e7-b4a9-456b-8bda-2120ff028a80 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=0ef930dc7b7a4fb29f7b32e464e9eb3b, role_id=d3975aa675c345d0974bbdede3f5e506)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:00.956242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3833e7-b4a9-456b-8bda-2120ff028a80 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:00.980426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 535/1072] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:53:00 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/0ef930dc7b7a4fb29f7b32e464e9eb3b/roles/d3975aa675c345d0974bbdede3f5e506 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:53:01.761292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c36f6-3c3b-4600-9692-ce1ba15a9bb1 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:01.761850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c36f6-3c3b-4600-9692-ce1ba15a9bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:01.761850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c36f6-3c3b-4600-9692-ce1ba15a9bb1 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:01.763369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 538/1073] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:01.768347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63800f82-40dd-48fa-b13f-f607f494f5d4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:01.768772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63800f82-40dd-48fa-b13f-f607f494f5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:01.768960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63800f82-40dd-48fa-b13f-f607f494f5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:01.786853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-63800f82-40dd-48fa-b13f-f607f494f5d4 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:01.788043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-63800f82-40dd-48fa-b13f-f607f494f5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:01.831367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:01.858930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 536/1074] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:01.881926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c36f6-3c3b-4600-9692-ce1ba15a9bb1 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:01.934020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:01.990390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c36f6-3c3b-4600-9692-ce1ba15a9bb1 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:02.026868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c008c3-26a9-4de0-863b-3af18bb9c01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:02.027589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c008c3-26a9-4de0-863b-3af18bb9c01e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:02.027873 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c008c3-26a9-4de0-863b-3af18bb9c01e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:02.028149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c008c3-26a9-4de0-863b-3af18bb9c01e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:02.039180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c008c3-26a9-4de0-863b-3af18bb9c01e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-b0c008c3-26a9-4de0-863b-3af18bb9c01e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:53:02.046723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:02.047679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:02.049139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:53:02.050111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 539/1075] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:53:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:02.054475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63800f82-40dd-48fa-b13f-f607f494f5d4 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:02.160361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63800f82-40dd-48fa-b13f-f607f494f5d4 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:02.196515 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e17bc-3a50-4bd9-a3e4-3262635bc6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:02.197188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e17bc-3a50-4bd9-a3e4-3262635bc6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:02.197470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e17bc-3a50-4bd9-a3e4-3262635bc6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:02.197744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e17bc-3a50-4bd9-a3e4-3262635bc6b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:02.198485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e17bc-3a50-4bd9-a3e4-3262635bc6b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:02.198790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e17bc-3a50-4bd9-a3e4-3262635bc6b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:02.200184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e17bc-3a50-4bd9-a3e4-3262635bc6b7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:02.213659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 537/1076] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:53:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:02.215160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c008c3-26a9-4de0-863b-3af18bb9c01e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:02.326861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c008c3-26a9-4de0-863b-3af18bb9c01e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:02.329244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:02.363771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4981207-366c-4f55-86f2-a16ca153a175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:02.364973 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4981207-366c-4f55-86f2-a16ca153a175 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:02.365294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4981207-366c-4f55-86f2-a16ca153a175 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:02.365541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4981207-366c-4f55-86f2-a16ca153a175 None admin] PATH_INFO: `/v3/users/df1e72a7024f43fe837ade3cbf393411` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:02.366736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4981207-366c-4f55-86f2-a16ca153a175 None admin] RBAC: Authorizing `identity:get_user(user_id=df1e72a7024f43fe837ade3cbf393411)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:02.368357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4981207-366c-4f55-86f2-a16ca153a175 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:02.372474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 540/1077] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:02 2022] GET /identity/v3/users/df1e72a7024f43fe837ade3cbf393411 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:02.375838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e17bc-3a50-4bd9-a3e4-3262635bc6b7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:02.418616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:02.481720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e17bc-3a50-4bd9-a3e4-3262635bc6b7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:02.515877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ced985-2969-4b91-96a8-d34266a6d2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:02.516767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ced985-2969-4b91-96a8-d34266a6d2be None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:02.517050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ced985-2969-4b91-96a8-d34266a6d2be None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:02.517322 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ced985-2969-4b91-96a8-d34266a6d2be None admin] PATH_INFO: `/v3/projects/a8579efc8c884a66b99f031edb4065ca` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:02.518496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ced985-2969-4b91-96a8-d34266a6d2be None admin] RBAC: Authorizing `identity:get_project(project_id=a8579efc8c884a66b99f031edb4065ca)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:02.520575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ced985-2969-4b91-96a8-d34266a6d2be None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:02.522673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 538/1078] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:02 2022] GET /identity/v3/projects/a8579efc8c884a66b99f031edb4065ca => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:02.528434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4981207-366c-4f55-86f2-a16ca153a175 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:02.625788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2c379-cb36-47dd-826f-874f3395fa43 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:02.626116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2c379-cb36-47dd-826f-874f3395fa43 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:02.626520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2c379-cb36-47dd-826f-874f3395fa43 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:02.628072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 539/1079] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:53:02.630030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4981207-366c-4f55-86f2-a16ca153a175 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:02.633109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff3cf-bc28-457d-9512-8c5e7ad435d6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:02.633399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff3cf-bc28-457d-9512-8c5e7ad435d6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:02.633677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff3cf-bc28-457d-9512-8c5e7ad435d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:02.651367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c06ff3cf-bc28-457d-9512-8c5e7ad435d6 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:02.652602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c06ff3cf-bc28-457d-9512-8c5e7ad435d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:02.666772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbb037-72a4-44a4-b0de-097a3a69269b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:02.667666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbb037-72a4-44a4-b0de-097a3a69269b None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:02.667954 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbb037-72a4-44a4-b0de-097a3a69269b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:02.668232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbb037-72a4-44a4-b0de-097a3a69269b None admin] PATH_INFO: `/v3/projects/a8579efc8c884a66b99f031edb4065ca/users/df1e72a7024f43fe837ade3cbf393411/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:02.670241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bbb037-72a4-44a4-b0de-097a3a69269b None admin] RBAC: Authorizing `identity:create_grant(project_id=a8579efc8c884a66b99f031edb4065ca, user_id=df1e72a7024f43fe837ade3cbf393411, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:02.671675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bbb037-72a4-44a4-b0de-097a3a69269b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:02.685324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 541/1080] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:53:02 2022] PUT /identity/v3/projects/a8579efc8c884a66b99f031edb4065ca/users/df1e72a7024f43fe837ade3cbf393411/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:53:02.740754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 540/1081] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:53:02.767512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbb037-72a4-44a4-b0de-097a3a69269b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:02.805983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:02.891699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbb037-72a4-44a4-b0de-097a3a69269b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:02.936066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8f2a2-afa4-4fd2-a282-baa4ecf19616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:02.936756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8f2a2-afa4-4fd2-a282-baa4ecf19616 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:02.937048 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8f2a2-afa4-4fd2-a282-baa4ecf19616 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:02.937326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8f2a2-afa4-4fd2-a282-baa4ecf19616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:03.007872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8f2a2-afa4-4fd2-a282-baa4ecf19616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:03.009383 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8f2a2-afa4-4fd2-a282-baa4ecf19616 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:03.041358 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:03.126691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 542/1082] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:53:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:53:03.127716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ff3cf-bc28-457d-9512-8c5e7ad435d6 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:03.164681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:03.222656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db1216-2273-43d6-9db4-946078a88932 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:03.223029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db1216-2273-43d6-9db4-946078a88932 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:03.223325 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db1216-2273-43d6-9db4-946078a88932 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:03.224966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 543/1083] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:53:03.229590 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d5e36-20c3-4747-969c-414743dd6ab2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:03.229881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d5e36-20c3-4747-969c-414743dd6ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:03.230153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d5e36-20c3-4747-969c-414743dd6ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:03.235089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ff3cf-bc28-457d-9512-8c5e7ad435d6 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:03.247271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-342d5e36-20c3-4747-969c-414743dd6ab2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:03.248494 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-342d5e36-20c3-4747-969c-414743dd6ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:03.273129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2e3ca-52e7-4a60-828f-eaabe52dcf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:03.273927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2e3ca-52e7-4a60-828f-eaabe52dcf6d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:03.274213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2e3ca-52e7-4a60-828f-eaabe52dcf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:03.274493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2e3ca-52e7-4a60-828f-eaabe52dcf6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:03.285403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2e3ca-52e7-4a60-828f-eaabe52dcf6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:03.286872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2e3ca-52e7-4a60-828f-eaabe52dcf6d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d0a2e3ca-52e7-4a60-828f-eaabe52dcf6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:53:03.307145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:03.308276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:03.309403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:53:03.309403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:53:03.309403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:53:03.309403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:53:03.312538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 541/1084] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:03.314153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2e3ca-52e7-4a60-828f-eaabe52dcf6d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:03.321723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 544/1085] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:53:03.344088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d5e36-20c3-4747-969c-414743dd6ab2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:03.404262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:03.408272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:03.419055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2e3ca-52e7-4a60-828f-eaabe52dcf6d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:03.453967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d5e36-20c3-4747-969c-414743dd6ab2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:03.457083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a346-a360-49dc-b1c7-e1befc9da951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:03.458331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a346-a360-49dc-b1c7-e1befc9da951 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:03.458614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a346-a360-49dc-b1c7-e1befc9da951 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:03.458890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a346-a360-49dc-b1c7-e1befc9da951 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:03.459608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a346-a360-49dc-b1c7-e1befc9da951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:03.459914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a346-a360-49dc-b1c7-e1befc9da951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:03.461320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a346-a360-49dc-b1c7-e1befc9da951 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:03.473342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 542/1086] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:03.491681 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1266b4-5be4-4d3b-94ea-e24014e19511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:03.492565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1266b4-5be4-4d3b-94ea-e24014e19511 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:03.492849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1266b4-5be4-4d3b-94ea-e24014e19511 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:03.493117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1266b4-5be4-4d3b-94ea-e24014e19511 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:03.506314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1266b4-5be4-4d3b-94ea-e24014e19511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-4e1266b4-5be4-4d3b-94ea-e24014e19511 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:53:03.514750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:03.515769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 08 01:53:03.517062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:53:03.520796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 545/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Jun 8 01:53:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:53:03.522523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a346-a360-49dc-b1c7-e1befc9da951 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:03.633591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a346-a360-49dc-b1c7-e1befc9da951 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:03.673934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e43d8-32d6-428f-9db7-816344216d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:03.674530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e43d8-32d6-428f-9db7-816344216d1b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:03.674817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e43d8-32d6-428f-9db7-816344216d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:03.675089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e43d8-32d6-428f-9db7-816344216d1b None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:03.675828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e43d8-32d6-428f-9db7-816344216d1b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:03.676194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e43d8-32d6-428f-9db7-816344216d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:03.677588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e43d8-32d6-428f-9db7-816344216d1b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:03.686706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 543/1088] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jun 8 01:53:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:03.692901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1266b4-5be4-4d3b-94ea-e24014e19511 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:03.798893 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1266b4-5be4-4d3b-94ea-e24014e19511 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:03.831573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:03.844171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea197b-689e-4e73-b96b-56e3e6f655ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:03.844998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea197b-689e-4e73-b96b-56e3e6f655ed None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:03.845284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea197b-689e-4e73-b96b-56e3e6f655ed None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:03.845561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea197b-689e-4e73-b96b-56e3e6f655ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:03.856834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea197b-689e-4e73-b96b-56e3e6f655ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:03.858288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea197b-689e-4e73-b96b-56e3e6f655ed None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-58ea197b-689e-4e73-b96b-56e3e6f655ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:53:03.875640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:03.876734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:03.882261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:53:03.882261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:53:03.882261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:53:03.882261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:53:03.882261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 546/1089] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:03.883694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e43d8-32d6-428f-9db7-816344216d1b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:03.896359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:03.992581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e43d8-32d6-428f-9db7-816344216d1b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:04.025262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62a41a-44e2-43a6-80de-7109cd0f9f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:04.025917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62a41a-44e2-43a6-80de-7109cd0f9f30 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:04.026199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62a41a-44e2-43a6-80de-7109cd0f9f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:04.026473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62a41a-44e2-43a6-80de-7109cd0f9f30 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:04.027172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62a41a-44e2-43a6-80de-7109cd0f9f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:04.027475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62a41a-44e2-43a6-80de-7109cd0f9f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:04.028910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62a41a-44e2-43a6-80de-7109cd0f9f30 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:04.041189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 544/1090] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:04.046679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea197b-689e-4e73-b96b-56e3e6f655ed None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:04.149361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea197b-689e-4e73-b96b-56e3e6f655ed None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:04.182438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de961b85-4b1b-432c-9cda-a1a791479ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:04.183217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de961b85-4b1b-432c-9cda-a1a791479ebf None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:04.183497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de961b85-4b1b-432c-9cda-a1a791479ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:04.183771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de961b85-4b1b-432c-9cda-a1a791479ebf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:04.190979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de961b85-4b1b-432c-9cda-a1a791479ebf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:04.192431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de961b85-4b1b-432c-9cda-a1a791479ebf None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-de961b85-4b1b-432c-9cda-a1a791479ebf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:53:04.205926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:04.207051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:04.208943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:04.208943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:04.208943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:53:04.208943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:53:04.208943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:53:04.208943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:53:04.208943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:53:04.208943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:53:04.208943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 08 01:53:04.208943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:53:04.212240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 547/1091] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Jun 8 01:53:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:04.213212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62a41a-44e2-43a6-80de-7109cd0f9f30 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:04.280219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:04.312613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62a41a-44e2-43a6-80de-7109cd0f9f30 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:04.345886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172f092-b642-4afb-b19e-b5bd361d7dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:04.346556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172f092-b642-4afb-b19e-b5bd361d7dbb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:04.346843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172f092-b642-4afb-b19e-b5bd361d7dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:04.347122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172f092-b642-4afb-b19e-b5bd361d7dbb None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:04.348544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172f092-b642-4afb-b19e-b5bd361d7dbb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:04.348897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172f092-b642-4afb-b19e-b5bd361d7dbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:04.350575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172f092-b642-4afb-b19e-b5bd361d7dbb None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:04.367898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 545/1092] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Jun 8 01:53:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:04.373550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de961b85-4b1b-432c-9cda-a1a791479ebf None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:04.411769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:04.477485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de961b85-4b1b-432c-9cda-a1a791479ebf None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:04.510631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3747a-8647-4be4-9e5b-61d4882db267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:04.511271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3747a-8647-4be4-9e5b-61d4882db267 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:04.511550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3747a-8647-4be4-9e5b-61d4882db267 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:04.511829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3747a-8647-4be4-9e5b-61d4882db267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:04.578461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3747a-8647-4be4-9e5b-61d4882db267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:04.579829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3747a-8647-4be4-9e5b-61d4882db267 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:04.612411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:04.685539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 548/1093] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:53:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:53:04.691161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172f092-b642-4afb-b19e-b5bd361d7dbb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:04.794453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172f092-b642-4afb-b19e-b5bd361d7dbb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:04.820677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:04.829060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59536842-8845-4e87-b95e-3ea81b0f2dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:04.829792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536842-8845-4e87-b95e-3ea81b0f2dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:04.830070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536842-8845-4e87-b95e-3ea81b0f2dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:04.830341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59536842-8845-4e87-b95e-3ea81b0f2dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:04.841448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536842-8845-4e87-b95e-3ea81b0f2dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:04.842901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59536842-8845-4e87-b95e-3ea81b0f2dc6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-59536842-8845-4e87-b95e-3ea81b0f2dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:53:04.860861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:04.861930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:04.867361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:53:04.867361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:53:04.867361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:53:04.867361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:53:04.867361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 546/1094] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:04.867652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3747a-8647-4be4-9e5b-61d4882db267 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:04.957027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9455c-a752-4fd6-bcfb-de3ea1908ba9 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:04.957848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9455c-a752-4fd6-bcfb-de3ea1908ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:04.958710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9455c-a752-4fd6-bcfb-de3ea1908ba9 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:04.960073 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 547/1095] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:53:04.965093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0153ae1-8af4-4f8a-b60f-09c6839a5c18 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:04.965384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0153ae1-8af4-4f8a-b60f-09c6839a5c18 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:04.965665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0153ae1-8af4-4f8a-b60f-09c6839a5c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:04.968898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3747a-8647-4be4-9e5b-61d4882db267 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:04.984471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d0153ae1-8af4-4f8a-b60f-09c6839a5c18 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:04.985649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d0153ae1-8af4-4f8a-b60f-09c6839a5c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:05.003560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6f328-a856-4058-a196-78779920cf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:05.004237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6f328-a856-4058-a196-78779920cf85 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:05.004552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6f328-a856-4058-a196-78779920cf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:05.004834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6f328-a856-4058-a196-78779920cf85 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:05.005542 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6f328-a856-4058-a196-78779920cf85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:05.005846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6f328-a856-4058-a196-78779920cf85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:05.007205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6f328-a856-4058-a196-78779920cf85 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:05.008659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:05.019332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 549/1096] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:05.025422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6f328-a856-4058-a196-78779920cf85 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:05.058311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 548/1097] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:53:05.081581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0153ae1-8af4-4f8a-b60f-09c6839a5c18 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:05.123529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6f328-a856-4058-a196-78779920cf85 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:05.156224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe3e67-bf5b-4323-aa8e-c2ea085545f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:05.157040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe3e67-bf5b-4323-aa8e-c2ea085545f2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:05.157326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe3e67-bf5b-4323-aa8e-c2ea085545f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:05.157599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe3e67-bf5b-4323-aa8e-c2ea085545f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:05.165606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe3e67-bf5b-4323-aa8e-c2ea085545f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:05.167045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe3e67-bf5b-4323-aa8e-c2ea085545f2 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-40fe3e67-bf5b-4323-aa8e-c2ea085545f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:53:05.181754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:05.182838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:05.183775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:53:05.183775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:53:05.183775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:53:05.183775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:53:05.183775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:53:05.183775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:53:05.183775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:53:05.183775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:53:05.183775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 550/1098] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:53:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:53:05.185194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0153ae1-8af4-4f8a-b60f-09c6839a5c18 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:05.188692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe3e67-bf5b-4323-aa8e-c2ea085545f2 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:05.196567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:05.220140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0303ce-f326-4e18-831d-afc6fa7ea1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:05.220956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0303ce-f326-4e18-831d-afc6fa7ea1bb None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:05.221237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0303ce-f326-4e18-831d-afc6fa7ea1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:05.221511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0303ce-f326-4e18-831d-afc6fa7ea1bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:05.232634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0303ce-f326-4e18-831d-afc6fa7ea1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-cd0303ce-f326-4e18-831d-afc6fa7ea1bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:53:05.241228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:05.242177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:53:05.243053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 549/1099] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:53:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:53:05.245580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:05.248447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0303ce-f326-4e18-831d-afc6fa7ea1bb None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:05.291967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe3e67-bf5b-4323-aa8e-c2ea085545f2 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:05.325812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526c743-13ea-4d31-99d1-8eadf0b1ffa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:05.326466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526c743-13ea-4d31-99d1-8eadf0b1ffa6 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:05.326749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526c743-13ea-4d31-99d1-8eadf0b1ffa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:05.327023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526c743-13ea-4d31-99d1-8eadf0b1ffa6 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:05.327805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526c743-13ea-4d31-99d1-8eadf0b1ffa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:05.328113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526c743-13ea-4d31-99d1-8eadf0b1ffa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:05.329580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526c743-13ea-4d31-99d1-8eadf0b1ffa6 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:05.341624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 551/1100] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:53:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:05.347644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526c743-13ea-4d31-99d1-8eadf0b1ffa6 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:05.348347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0303ce-f326-4e18-831d-afc6fa7ea1bb None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:05.383372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086a93d-b942-496a-bf47-a7bbe4598726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:05.384046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086a93d-b942-496a-bf47-a7bbe4598726 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:05.384331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086a93d-b942-496a-bf47-a7bbe4598726 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:05.384644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086a93d-b942-496a-bf47-a7bbe4598726 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:05.385387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086a93d-b942-496a-bf47-a7bbe4598726 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:05.385694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086a93d-b942-496a-bf47-a7bbe4598726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:05.387083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086a93d-b942-496a-bf47-a7bbe4598726 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:05.396198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 550/1101] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:53:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:05.401884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086a93d-b942-496a-bf47-a7bbe4598726 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:05.432134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:05.454779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526c743-13ea-4d31-99d1-8eadf0b1ffa6 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:05.468707 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:05.490300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198763-4e59-4916-8f6b-c65f62b5a1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:05.490959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198763-4e59-4916-8f6b-c65f62b5a1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:05.491245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198763-4e59-4916-8f6b-c65f62b5a1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:05.491524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198763-4e59-4916-8f6b-c65f62b5a1e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:05.492352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e198763-4e59-4916-8f6b-c65f62b5a1e1 None admin] RBAC: Adding query filter params (role.id=d3975aa675c345d0974bbdede3f5e506, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:05.492711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e198763-4e59-4916-8f6b-c65f62b5a1e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:05.494160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e198763-4e59-4916-8f6b-c65f62b5a1e1 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:05.500068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 552/1102] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:53:05 2022] GET /identity/v3/role_assignments?user.id=0ef930dc7b7a4fb29f7b32e464e9eb3b&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=d3975aa675c345d0974bbdede3f5e506 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:05.511112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086a93d-b942-496a-bf47-a7bbe4598726 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:05.548814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578a5f8-f7eb-44e9-b2d5-b1593f5f975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:05.549695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578a5f8-f7eb-44e9-b2d5-b1593f5f975b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:05.549976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578a5f8-f7eb-44e9-b2d5-b1593f5f975b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:05.550249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578a5f8-f7eb-44e9-b2d5-b1593f5f975b None admin] PATH_INFO: `/v3/users/df1e72a7024f43fe837ade3cbf393411` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:05.551516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578a5f8-f7eb-44e9-b2d5-b1593f5f975b None admin] RBAC: Authorizing `identity:get_user(user_id=df1e72a7024f43fe837ade3cbf393411)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:05.553062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578a5f8-f7eb-44e9-b2d5-b1593f5f975b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:05.554738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 551/1103] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:05 2022] GET /identity/v3/users/df1e72a7024f43fe837ade3cbf393411 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:05.560709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198763-4e59-4916-8f6b-c65f62b5a1e1 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:05.670326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198763-4e59-4916-8f6b-c65f62b5a1e1 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:05.704844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d075c09-e111-484a-82fa-18ce4b2f5d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:05.705507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d075c09-e111-484a-82fa-18ce4b2f5d11 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:05.705794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d075c09-e111-484a-82fa-18ce4b2f5d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:05.706079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d075c09-e111-484a-82fa-18ce4b2f5d11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:05.755113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:05.779399 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d075c09-e111-484a-82fa-18ce4b2f5d11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:05.780923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d075c09-e111-484a-82fa-18ce4b2f5d11 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:05.884515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 553/1104] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:53:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:53:05.890935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578a5f8-f7eb-44e9-b2d5-b1593f5f975b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:05.997838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:06.001051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578a5f8-f7eb-44e9-b2d5-b1593f5f975b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:06.038375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe68eb-9400-4842-9ae1-183c10c8366d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:06.039245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe68eb-9400-4842-9ae1-183c10c8366d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:06.039530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe68eb-9400-4842-9ae1-183c10c8366d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:06.039806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe68eb-9400-4842-9ae1-183c10c8366d None admin] PATH_INFO: `/v3/projects/a8579efc8c884a66b99f031edb4065ca` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:06.041026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe68eb-9400-4842-9ae1-183c10c8366d None admin] RBAC: Authorizing `identity:get_project(project_id=a8579efc8c884a66b99f031edb4065ca)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:06.042478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe68eb-9400-4842-9ae1-183c10c8366d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:06.044182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 552/1105] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:05 2022] GET /identity/v3/projects/a8579efc8c884a66b99f031edb4065ca => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:06.050755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d075c09-e111-484a-82fa-18ce4b2f5d11 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:06.129799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:06.151771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d075c09-e111-484a-82fa-18ce4b2f5d11 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:06.186157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53414d-e874-4ee8-af6d-1a497a1a4d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:06.186817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53414d-e874-4ee8-af6d-1a497a1a4d16 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:06.187098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53414d-e874-4ee8-af6d-1a497a1a4d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:06.187371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53414d-e874-4ee8-af6d-1a497a1a4d16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:06.188199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53414d-e874-4ee8-af6d-1a497a1a4d16 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=a8579efc8c884a66b99f031edb4065ca, user.id=df1e72a7024f43fe837ade3cbf393411) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:06.188538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53414d-e874-4ee8-af6d-1a497a1a4d16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:06.189967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53414d-e874-4ee8-af6d-1a497a1a4d16 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:06.196462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 554/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:53:06 2022] GET /identity/v3/role_assignments?user.id=df1e72a7024f43fe837ade3cbf393411&scope.project.id=a8579efc8c884a66b99f031edb4065ca&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:08.460501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8639b-ce02-4402-8d1a-6569559cac94 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:08.461033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8639b-ce02-4402-8d1a-6569559cac94 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:08.461033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8639b-ce02-4402-8d1a-6569559cac94 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:08.462403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 553/1107] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:08.468621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c43380-52e0-4c8a-8032-41f83d026408 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:08.469022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c43380-52e0-4c8a-8032-41f83d026408 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:08.469187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c43380-52e0-4c8a-8032-41f83d026408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:08.489717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-42c43380-52e0-4c8a-8032-41f83d026408 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:08.490979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-42c43380-52e0-4c8a-8032-41f83d026408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:08.557195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 555/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:08.582315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8639b-ce02-4402-8d1a-6569559cac94 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:08.684556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8639b-ce02-4402-8d1a-6569559cac94 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:08.717134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68127240-5a94-4c57-86e5-c73fb68a932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:08.717857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68127240-5a94-4c57-86e5-c73fb68a932b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:08.718141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68127240-5a94-4c57-86e5-c73fb68a932b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:08.718413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68127240-5a94-4c57-86e5-c73fb68a932b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:08.719572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68127240-5a94-4c57-86e5-c73fb68a932b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:08.721046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68127240-5a94-4c57-86e5-c73fb68a932b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:08.723278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 554/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:53:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:08.728832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c43380-52e0-4c8a-8032-41f83d026408 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:08.806810 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:08.835928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c43380-52e0-4c8a-8032-41f83d026408 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:08.871852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13f6d4-f8fc-4278-8fc9-a2a1a273e619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:08.872547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13f6d4-f8fc-4278-8fc9-a2a1a273e619 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:08.872834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13f6d4-f8fc-4278-8fc9-a2a1a273e619 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:08.873113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13f6d4-f8fc-4278-8fc9-a2a1a273e619 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:08.873827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13f6d4-f8fc-4278-8fc9-a2a1a273e619 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:08.875274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13f6d4-f8fc-4278-8fc9-a2a1a273e619 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:08.894545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 556/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:53:08 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 01:53:11.143363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913a356-2509-45fd-9e49-18628d6e5c8e None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:11.143886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913a356-2509-45fd-9e49-18628d6e5c8e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:11.143971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913a356-2509-45fd-9e49-18628d6e5c8e None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:11.146015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 555/1111] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:11.151256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb62af-c775-48ab-bb9c-1a79759355a6 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:11.151652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb62af-c775-48ab-bb9c-1a79759355a6 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:11.151823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb62af-c775-48ab-bb9c-1a79759355a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:11.168967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ebbb62af-c775-48ab-bb9c-1a79759355a6 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:11.170085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ebbb62af-c775-48ab-bb9c-1a79759355a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:11.234997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 557/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:11.258232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913a356-2509-45fd-9e49-18628d6e5c8e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:11.294190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:11.356883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913a356-2509-45fd-9e49-18628d6e5c8e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:11.392178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf985f4-93a1-42b6-9b49-f29fc21d2223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:11.393259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf985f4-93a1-42b6-9b49-f29fc21d2223 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:11.393539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf985f4-93a1-42b6-9b49-f29fc21d2223 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:11.393813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf985f4-93a1-42b6-9b49-f29fc21d2223 None admin] PATH_INFO: `/v3/roles/c45c9440d2b8447ab50b814c6be35bf1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:11.395291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf985f4-93a1-42b6-9b49-f29fc21d2223 None admin] RBAC: Authorizing `identity:get_role(role_id=c45c9440d2b8447ab50b814c6be35bf1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:11.396685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf985f4-93a1-42b6-9b49-f29fc21d2223 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:11.398054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 556/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:11 2022] GET /identity/v3/roles/c45c9440d2b8447ab50b814c6be35bf1 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:11.403861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb62af-c775-48ab-bb9c-1a79759355a6 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:11.461642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:11.502854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb62af-c775-48ab-bb9c-1a79759355a6 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:11.537279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2fc3f-c2d6-4a30-8e51-6c0f66a6f98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:11.538130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2fc3f-c2d6-4a30-8e51-6c0f66a6f98b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:11.538411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2fc3f-c2d6-4a30-8e51-6c0f66a6f98b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:11.538685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2fc3f-c2d6-4a30-8e51-6c0f66a6f98b None admin] PATH_INFO: `/v3/users/e3c4f02082ef4aa49a962b26d5faa864` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:11.551179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2fc3f-c2d6-4a30-8e51-6c0f66a6f98b None admin] RBAC: Authorizing `identity:get_user(user_id=e3c4f02082ef4aa49a962b26d5faa864)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:11.552664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2fc3f-c2d6-4a30-8e51-6c0f66a6f98b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:11.554278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 558/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:11 2022] GET /identity/v3/users/e3c4f02082ef4aa49a962b26d5faa864 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:11.560140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf985f4-93a1-42b6-9b49-f29fc21d2223 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:11.658858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf985f4-93a1-42b6-9b49-f29fc21d2223 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:11.662002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:11.699563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8d56-aaa0-48cd-9d3d-3c06ef092c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:11.700214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8d56-aaa0-48cd-9d3d-3c06ef092c33 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:11.700536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8d56-aaa0-48cd-9d3d-3c06ef092c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:11.700816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e8d56-aaa0-48cd-9d3d-3c06ef092c33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:11.766263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e8d56-aaa0-48cd-9d3d-3c06ef092c33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:11.767712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e8d56-aaa0-48cd-9d3d-3c06ef092c33 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:11.863386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 557/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:53:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:53:11.869311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2fc3f-c2d6-4a30-8e51-6c0f66a6f98b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:11.970766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2fc3f-c2d6-4a30-8e51-6c0f66a6f98b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:11.986647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:12.004955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f810b-7c69-4712-9979-e918d5bd53cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:12.005814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f810b-7c69-4712-9979-e918d5bd53cc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:12.006099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f810b-7c69-4712-9979-e918d5bd53cc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:12.006375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f810b-7c69-4712-9979-e918d5bd53cc None admin] PATH_INFO: `/v3/projects/a8579efc8c884a66b99f031edb4065ca` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:12.007550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f810b-7c69-4712-9979-e918d5bd53cc None admin] RBAC: Authorizing `identity:get_project(project_id=a8579efc8c884a66b99f031edb4065ca)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:12.009033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f810b-7c69-4712-9979-e918d5bd53cc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:12.010712 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 559/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:11 2022] GET /identity/v3/projects/a8579efc8c884a66b99f031edb4065ca => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:12.016882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8d56-aaa0-48cd-9d3d-3c06ef092c33 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:12.020566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:12.113489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e8d56-aaa0-48cd-9d3d-3c06ef092c33 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:12.146256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c7ade-3755-49bd-96c4-6ac27a38050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:12.146899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c7ade-3755-49bd-96c4-6ac27a38050c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:12.147182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c7ade-3755-49bd-96c4-6ac27a38050c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:12.147456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c7ade-3755-49bd-96c4-6ac27a38050c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:12.148270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c7ade-3755-49bd-96c4-6ac27a38050c None admin] RBAC: Adding query filter params (role.id=c45c9440d2b8447ab50b814c6be35bf1, scope.project.id=a8579efc8c884a66b99f031edb4065ca, user.id=e3c4f02082ef4aa49a962b26d5faa864) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:12.148609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c7ade-3755-49bd-96c4-6ac27a38050c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:12.150060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c7ade-3755-49bd-96c4-6ac27a38050c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:12.155498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 558/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:53:12 2022] GET /identity/v3/role_assignments?user.id=e3c4f02082ef4aa49a962b26d5faa864&scope.project.id=a8579efc8c884a66b99f031edb4065ca&role.id=c45c9440d2b8447ab50b814c6be35bf1 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:14.335828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd2cf6-e621-419d-ac44-67989241ec83 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:14.336350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd2cf6-e621-419d-ac44-67989241ec83 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:14.336350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd2cf6-e621-419d-ac44-67989241ec83 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:14.337746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 560/1118] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:14.342954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00e814-94b7-4179-990d-57137fa5dcd2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:14.343354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00e814-94b7-4179-990d-57137fa5dcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:14.343508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00e814-94b7-4179-990d-57137fa5dcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:14.361329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2e00e814-94b7-4179-990d-57137fa5dcd2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:14.362510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2e00e814-94b7-4179-990d-57137fa5dcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:14.422770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:14.428875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 559/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:14.452515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd2cf6-e621-419d-ac44-67989241ec83 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:14.552618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd2cf6-e621-419d-ac44-67989241ec83 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:14.585624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eee7b5-7836-4f27-ba39-ed515e76235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:14.586516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eee7b5-7836-4f27-ba39-ed515e76235e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:14.586745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eee7b5-7836-4f27-ba39-ed515e76235e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:14.587019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eee7b5-7836-4f27-ba39-ed515e76235e None admin] PATH_INFO: `/v3/roles/c45c9440d2b8447ab50b814c6be35bf1` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:14.588510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eee7b5-7836-4f27-ba39-ed515e76235e None admin] RBAC: Authorizing `identity:get_role(role_id=c45c9440d2b8447ab50b814c6be35bf1)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:14.589882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eee7b5-7836-4f27-ba39-ed515e76235e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:14.591225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 561/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:14 2022] GET /identity/v3/roles/c45c9440d2b8447ab50b814c6be35bf1 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:14.597856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00e814-94b7-4179-990d-57137fa5dcd2 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:14.694131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00e814-94b7-4179-990d-57137fa5dcd2 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:14.727401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf8abf-50f0-4b00-b01e-915d4531e07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:14.728253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf8abf-50f0-4b00-b01e-915d4531e07b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:14.728566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf8abf-50f0-4b00-b01e-915d4531e07b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:14.728843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf8abf-50f0-4b00-b01e-915d4531e07b None admin] PATH_INFO: `/v3/users/e3c4f02082ef4aa49a962b26d5faa864` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:14.730006 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf8abf-50f0-4b00-b01e-915d4531e07b None admin] RBAC: Authorizing `identity:get_user(user_id=e3c4f02082ef4aa49a962b26d5faa864)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:14.731434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf8abf-50f0-4b00-b01e-915d4531e07b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:14.733060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 560/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:14 2022] GET /identity/v3/users/e3c4f02082ef4aa49a962b26d5faa864 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:14.738664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eee7b5-7836-4f27-ba39-ed515e76235e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:14.790322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:14.835324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eee7b5-7836-4f27-ba39-ed515e76235e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:14.868268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b003494-232e-4ee2-a02c-bb738d50c588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:14.869081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b003494-232e-4ee2-a02c-bb738d50c588 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:14.869365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b003494-232e-4ee2-a02c-bb738d50c588 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:14.869641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b003494-232e-4ee2-a02c-bb738d50c588 None admin] PATH_INFO: `/v3/projects/a8579efc8c884a66b99f031edb4065ca` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:14.870807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b003494-232e-4ee2-a02c-bb738d50c588 None admin] RBAC: Authorizing `identity:get_project(project_id=a8579efc8c884a66b99f031edb4065ca)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:14.872230 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b003494-232e-4ee2-a02c-bb738d50c588 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:14.873918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 562/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:14 2022] GET /identity/v3/projects/a8579efc8c884a66b99f031edb4065ca => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:14.880128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf8abf-50f0-4b00-b01e-915d4531e07b None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:14.945568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:14.979268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf8abf-50f0-4b00-b01e-915d4531e07b None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:15.011903 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fef66-4845-4483-a7bf-555cabc14a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:15.012809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fef66-4845-4483-a7bf-555cabc14a0a None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:15.013094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fef66-4845-4483-a7bf-555cabc14a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:15.013369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fef66-4845-4483-a7bf-555cabc14a0a None admin] PATH_INFO: `/v3/projects/a8579efc8c884a66b99f031edb4065ca/users/e3c4f02082ef4aa49a962b26d5faa864/roles/c45c9440d2b8447ab50b814c6be35bf1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:15.015551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fef66-4845-4483-a7bf-555cabc14a0a None admin] RBAC: Authorizing `identity:create_grant(project_id=a8579efc8c884a66b99f031edb4065ca, user_id=e3c4f02082ef4aa49a962b26d5faa864, role_id=c45c9440d2b8447ab50b814c6be35bf1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:15.017019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fef66-4845-4483-a7bf-555cabc14a0a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:15.030046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 561/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:53:14 2022] PUT /identity/v3/projects/a8579efc8c884a66b99f031edb4065ca/users/e3c4f02082ef4aa49a962b26d5faa864/roles/c45c9440d2b8447ab50b814c6be35bf1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:53:17.253068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3df837-c1fc-48ea-8969-8e7ea038d58c None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:17.253660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3df837-c1fc-48ea-8969-8e7ea038d58c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:17.253796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3df837-c1fc-48ea-8969-8e7ea038d58c None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:17.255158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 563/1124] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:17.260449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25c49e-19d3-464c-bbeb-9a79fccb2f56 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:17.260818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25c49e-19d3-464c-bbeb-9a79fccb2f56 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:17.260996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25c49e-19d3-464c-bbeb-9a79fccb2f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:17.277740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7f25c49e-19d3-464c-bbeb-9a79fccb2f56 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:17.278904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7f25c49e-19d3-464c-bbeb-9a79fccb2f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:17.344985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 562/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:17.370180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3df837-c1fc-48ea-8969-8e7ea038d58c None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:17.474004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3df837-c1fc-48ea-8969-8e7ea038d58c None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:17.494334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:17.508828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f15f5-abf5-4f5e-af1a-9be217705987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:17.509624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f15f5-abf5-4f5e-af1a-9be217705987 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:17.509904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f15f5-abf5-4f5e-af1a-9be217705987 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:17.510180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f15f5-abf5-4f5e-af1a-9be217705987 None admin] PATH_INFO: `/v3/roles/c45c9440d2b8447ab50b814c6be35bf1` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:17.511632 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f15f5-abf5-4f5e-af1a-9be217705987 None admin] RBAC: Authorizing `identity:get_role(role_id=c45c9440d2b8447ab50b814c6be35bf1)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:17.513178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f15f5-abf5-4f5e-af1a-9be217705987 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:17.514524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 564/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:17 2022] GET /identity/v3/roles/c45c9440d2b8447ab50b814c6be35bf1 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:17.520872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25c49e-19d3-464c-bbeb-9a79fccb2f56 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:17.576342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:17.622333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25c49e-19d3-464c-bbeb-9a79fccb2f56 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:17.656542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5607a0c-d6d2-4a1f-a78f-f6534d7ff2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:17.657414 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5607a0c-d6d2-4a1f-a78f-f6534d7ff2ec None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:17.657699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5607a0c-d6d2-4a1f-a78f-f6534d7ff2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:17.657980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5607a0c-d6d2-4a1f-a78f-f6534d7ff2ec None admin] PATH_INFO: `/v3/users/e3c4f02082ef4aa49a962b26d5faa864` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:17.659370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5607a0c-d6d2-4a1f-a78f-f6534d7ff2ec None admin] RBAC: Authorizing `identity:get_user(user_id=e3c4f02082ef4aa49a962b26d5faa864)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:17.661127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5607a0c-d6d2-4a1f-a78f-f6534d7ff2ec None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:17.663123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 563/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:17 2022] GET /identity/v3/users/e3c4f02082ef4aa49a962b26d5faa864 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:17.668922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f15f5-abf5-4f5e-af1a-9be217705987 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:17.766209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f15f5-abf5-4f5e-af1a-9be217705987 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:17.799184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d07377-47d1-45dd-9b41-b19fefca1b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:17.799816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d07377-47d1-45dd-9b41-b19fefca1b27 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:17.800099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d07377-47d1-45dd-9b41-b19fefca1b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:17.800927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d07377-47d1-45dd-9b41-b19fefca1b27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:17.854241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:17.866538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d07377-47d1-45dd-9b41-b19fefca1b27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:17.867987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d07377-47d1-45dd-9b41-b19fefca1b27 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:17.964619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 565/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:53:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:53:17.970393 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5607a0c-d6d2-4a1f-a78f-f6534d7ff2ec None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:18.070240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5607a0c-d6d2-4a1f-a78f-f6534d7ff2ec None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:18.093771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:18.114110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a3bd0-68e1-4b73-a2ef-9d8f863a68f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:18.114969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a3bd0-68e1-4b73-a2ef-9d8f863a68f7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:18.115257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a3bd0-68e1-4b73-a2ef-9d8f863a68f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:18.115541 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a3bd0-68e1-4b73-a2ef-9d8f863a68f7 None admin] PATH_INFO: `/v3/projects/a8579efc8c884a66b99f031edb4065ca` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:18.119572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a3bd0-68e1-4b73-a2ef-9d8f863a68f7 None admin] RBAC: Authorizing `identity:get_project(project_id=a8579efc8c884a66b99f031edb4065ca)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:18.121092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a3bd0-68e1-4b73-a2ef-9d8f863a68f7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:18.122787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 564/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:17 2022] GET /identity/v3/projects/a8579efc8c884a66b99f031edb4065ca => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:18.129397 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d07377-47d1-45dd-9b41-b19fefca1b27 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:18.215950 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:18.228175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d07377-47d1-45dd-9b41-b19fefca1b27 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:18.262418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643cc26-91d9-49e2-bde1-a62fbb6eb4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:18.263069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643cc26-91d9-49e2-bde1-a62fbb6eb4ff None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:18.263352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643cc26-91d9-49e2-bde1-a62fbb6eb4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:18.263628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643cc26-91d9-49e2-bde1-a62fbb6eb4ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:18.264475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643cc26-91d9-49e2-bde1-a62fbb6eb4ff None admin] RBAC: Adding query filter params (role.id=c45c9440d2b8447ab50b814c6be35bf1, scope.project.id=a8579efc8c884a66b99f031edb4065ca, user.id=e3c4f02082ef4aa49a962b26d5faa864) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:18.264789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643cc26-91d9-49e2-bde1-a62fbb6eb4ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:18.266222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643cc26-91d9-49e2-bde1-a62fbb6eb4ff None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:18.272085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 566/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:53:18 2022] GET /identity/v3/role_assignments?user.id=e3c4f02082ef4aa49a962b26d5faa864&scope.project.id=a8579efc8c884a66b99f031edb4065ca&role.id=c45c9440d2b8447ab50b814c6be35bf1 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:20.532713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47bbfd-cf1c-446b-b7ed-d291695e8f96 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:20.533268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47bbfd-cf1c-446b-b7ed-d291695e8f96 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:20.533268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47bbfd-cf1c-446b-b7ed-d291695e8f96 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:20.534932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 565/1131] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:20.540459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81f2ac-15a3-42a5-b5c3-733ab2f9814d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:20.540880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81f2ac-15a3-42a5-b5c3-733ab2f9814d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:20.541011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81f2ac-15a3-42a5-b5c3-733ab2f9814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:20.557815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-9a81f2ac-15a3-42a5-b5c3-733ab2f9814d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:20.559003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-9a81f2ac-15a3-42a5-b5c3-733ab2f9814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:20.622659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 567/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:53:20.646043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47bbfd-cf1c-446b-b7ed-d291695e8f96 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:20.745955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47bbfd-cf1c-446b-b7ed-d291695e8f96 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:20.778943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0724797-6090-43c6-930c-97bff30cd816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:20.779723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0724797-6090-43c6-930c-97bff30cd816 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:20.780006 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0724797-6090-43c6-930c-97bff30cd816 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:20.780280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0724797-6090-43c6-930c-97bff30cd816 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:20.781482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0724797-6090-43c6-930c-97bff30cd816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:20.782976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0724797-6090-43c6-930c-97bff30cd816 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:20.784878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 566/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:53:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:20.790621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81f2ac-15a3-42a5-b5c3-733ab2f9814d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:20.870350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:20.888203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81f2ac-15a3-42a5-b5c3-733ab2f9814d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:20.923182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011c167-e96c-4289-81d4-63a27d740110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:20.923840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c167-e96c-4289-81d4-63a27d740110 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:20.924122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c167-e96c-4289-81d4-63a27d740110 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:20.924411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c167-e96c-4289-81d4-63a27d740110 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:20.925863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011c167-e96c-4289-81d4-63a27d740110 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:20.927367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011c167-e96c-4289-81d4-63a27d740110 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:20.952963 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 568/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:53:20 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:53:23.193532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1202c-1bde-4c20-ae64-5536d72336cf None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:23.194045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1202c-1bde-4c20-ae64-5536d72336cf None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:23.194045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1202c-1bde-4c20-ae64-5536d72336cf None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:23.195354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 567/1135] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:23.200773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2ef6a-f232-4a1c-b98f-a46208c28f33 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:23.201179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2ef6a-f232-4a1c-b98f-a46208c28f33 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:23.201338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2ef6a-f232-4a1c-b98f-a46208c28f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:23.217927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-57b2ef6a-f232-4a1c-b98f-a46208c28f33 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:23.219106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-57b2ef6a-f232-4a1c-b98f-a46208c28f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:23.283748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 569/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:23.308233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1202c-1bde-4c20-ae64-5536d72336cf None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:23.366709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:23.407840 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1202c-1bde-4c20-ae64-5536d72336cf None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:23.441361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe0794-2a64-4354-be45-3d66dcbb4de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:23.442284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe0794-2a64-4354-be45-3d66dcbb4de4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:23.442565 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe0794-2a64-4354-be45-3d66dcbb4de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:23.442841 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe0794-2a64-4354-be45-3d66dcbb4de4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:23.444103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe0794-2a64-4354-be45-3d66dcbb4de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:23.445482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe0794-2a64-4354-be45-3d66dcbb4de4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:23.447063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 568/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:53:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:23.452654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2ef6a-f232-4a1c-b98f-a46208c28f33 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:23.514023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:23.553410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2ef6a-f232-4a1c-b98f-a46208c28f33 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:23.587270 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1816e7-6292-4134-91d3-c2aec123d51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:23.587924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1816e7-6292-4134-91d3-c2aec123d51c None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:23.588208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1816e7-6292-4134-91d3-c2aec123d51c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:23.588510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1816e7-6292-4134-91d3-c2aec123d51c None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:23.589217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1816e7-6292-4134-91d3-c2aec123d51c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:23.590655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1816e7-6292-4134-91d3-c2aec123d51c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:23.609604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 570/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:53:23 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:53:25.910636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaeea5-561e-4120-9475-8dba40d0a16f None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:25.911161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaeea5-561e-4120-9475-8dba40d0a16f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:25.911161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eaeea5-561e-4120-9475-8dba40d0a16f None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:25.912748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 569/1139] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:25.917912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2f845-e296-43d4-be38-02227c90cae8 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:25.918282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2f845-e296-43d4-be38-02227c90cae8 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:25.918475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2f845-e296-43d4-be38-02227c90cae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:25.935324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-eda2f845-e296-43d4-be38-02227c90cae8 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:25.936533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-eda2f845-e296-43d4-be38-02227c90cae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:26.001595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 571/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:26.025589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eaeea5-561e-4120-9475-8dba40d0a16f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:26.124917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eaeea5-561e-4120-9475-8dba40d0a16f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:26.150240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:26.158328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46038042-df66-4da5-be34-84c1bb12c170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:26.159662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46038042-df66-4da5-be34-84c1bb12c170 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:26.160031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46038042-df66-4da5-be34-84c1bb12c170 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:26.160309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46038042-df66-4da5-be34-84c1bb12c170 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:26.170301 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46038042-df66-4da5-be34-84c1bb12c170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-46038042-df66-4da5-be34-84c1bb12c170 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:53:26.177909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:26.179034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:53:26.179934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 570/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:53:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:26.185043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2f845-e296-43d4-be38-02227c90cae8 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:26.233328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:26.283362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2f845-e296-43d4-be38-02227c90cae8 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:26.317229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a968c4-89a2-4c04-ba8d-55c0d1eaf19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:26.317887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a968c4-89a2-4c04-ba8d-55c0d1eaf19e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:26.318174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a968c4-89a2-4c04-ba8d-55c0d1eaf19e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:26.318451 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a968c4-89a2-4c04-ba8d-55c0d1eaf19e None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:26.319192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a968c4-89a2-4c04-ba8d-55c0d1eaf19e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:26.319522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a968c4-89a2-4c04-ba8d-55c0d1eaf19e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:26.320940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a968c4-89a2-4c04-ba8d-55c0d1eaf19e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:26.328839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 572/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:53:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:26.335108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46038042-df66-4da5-be34-84c1bb12c170 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:26.441804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46038042-df66-4da5-be34-84c1bb12c170 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:26.475050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f894a-80a4-455a-9b38-d3dbd781574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:26.475919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f894a-80a4-455a-9b38-d3dbd781574f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:26.476194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f894a-80a4-455a-9b38-d3dbd781574f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:26.476492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f894a-80a4-455a-9b38-d3dbd781574f None admin] PATH_INFO: `/v3/users/5e6518c4f78d41e89e91fd57d8dfe116` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:26.489671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f894a-80a4-455a-9b38-d3dbd781574f None admin] RBAC: Authorizing `identity:get_user(user_id=5e6518c4f78d41e89e91fd57d8dfe116)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:26.491091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f894a-80a4-455a-9b38-d3dbd781574f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:26.492744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 571/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:26 2022] GET /identity/v3/users/5e6518c4f78d41e89e91fd57d8dfe116 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:26.498621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a968c4-89a2-4c04-ba8d-55c0d1eaf19e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:26.597825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a968c4-89a2-4c04-ba8d-55c0d1eaf19e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:26.609906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:26.631581 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a993e-0e56-4b57-bbc6-a47aef5f7b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:26.632216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a993e-0e56-4b57-bbc6-a47aef5f7b9c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:26.632527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a993e-0e56-4b57-bbc6-a47aef5f7b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:26.632803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a993e-0e56-4b57-bbc6-a47aef5f7b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:26.697844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a993e-0e56-4b57-bbc6-a47aef5f7b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:26.699229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a993e-0e56-4b57-bbc6-a47aef5f7b9c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:26.798823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 573/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:53:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:53:26.805333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f894a-80a4-455a-9b38-d3dbd781574f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:26.850901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:26.901870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f894a-80a4-455a-9b38-d3dbd781574f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:26.935105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fa8e8-5c62-4b55-86ee-a2a6fbc2589f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:26.935959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fa8e8-5c62-4b55-86ee-a2a6fbc2589f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:26.936238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fa8e8-5c62-4b55-86ee-a2a6fbc2589f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:26.936544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fa8e8-5c62-4b55-86ee-a2a6fbc2589f None admin] PATH_INFO: `/v3/projects/34e4b84982ee4f37b77fa6dfe14815bd` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:26.937702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fa8e8-5c62-4b55-86ee-a2a6fbc2589f None admin] RBAC: Authorizing `identity:get_project(project_id=34e4b84982ee4f37b77fa6dfe14815bd)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:26.939128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fa8e8-5c62-4b55-86ee-a2a6fbc2589f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:26.941009 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 572/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:26 2022] GET /identity/v3/projects/34e4b84982ee4f37b77fa6dfe14815bd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:26.947043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a993e-0e56-4b57-bbc6-a47aef5f7b9c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:26.960933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:27.046057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a993e-0e56-4b57-bbc6-a47aef5f7b9c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:27.079740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9303f8-9fd9-4c4b-b9d7-33a215906feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:27.080387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9303f8-9fd9-4c4b-b9d7-33a215906feb None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:27.080699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9303f8-9fd9-4c4b-b9d7-33a215906feb None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:27.081018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9303f8-9fd9-4c4b-b9d7-33a215906feb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:27.081841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9303f8-9fd9-4c4b-b9d7-33a215906feb None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=34e4b84982ee4f37b77fa6dfe14815bd, user.id=5e6518c4f78d41e89e91fd57d8dfe116) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:27.082154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9303f8-9fd9-4c4b-b9d7-33a215906feb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:27.083582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9303f8-9fd9-4c4b-b9d7-33a215906feb None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:27.088938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 574/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:53:26 2022] GET /identity/v3/role_assignments?user.id=5e6518c4f78d41e89e91fd57d8dfe116&scope.project.id=34e4b84982ee4f37b77fa6dfe14815bd&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:29.273842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c960c8c-bf3e-4bc6-970a-261a46de7693 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:29.274377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c960c8c-bf3e-4bc6-970a-261a46de7693 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:29.274377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c960c8c-bf3e-4bc6-970a-261a46de7693 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:29.275761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 573/1147] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:29.280939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb0ab4-fe82-4227-a7fd-1096b1cbd55d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:29.281372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb0ab4-fe82-4227-a7fd-1096b1cbd55d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:29.281505 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb0ab4-fe82-4227-a7fd-1096b1cbd55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:29.299318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-68cb0ab4-fe82-4227-a7fd-1096b1cbd55d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:29.301303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-68cb0ab4-fe82-4227-a7fd-1096b1cbd55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:29.365609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 575/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:29.388089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c960c8c-bf3e-4bc6-970a-261a46de7693 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:29.491122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c960c8c-bf3e-4bc6-970a-261a46de7693 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:29.506190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:29.525869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812a634-3b5e-4c6b-acd4-fc8918808276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:29.526593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812a634-3b5e-4c6b-acd4-fc8918808276 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:29.526872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812a634-3b5e-4c6b-acd4-fc8918808276 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:29.527140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812a634-3b5e-4c6b-acd4-fc8918808276 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:29.537620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812a634-3b5e-4c6b-acd4-fc8918808276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-f812a634-3b5e-4c6b-acd4-fc8918808276 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:53:29.545113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:29.546275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:53:29.547358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 574/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:53:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:29.553151 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb0ab4-fe82-4227-a7fd-1096b1cbd55d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:29.561096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:29.650431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb0ab4-fe82-4227-a7fd-1096b1cbd55d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:29.685089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0d757-de20-40d7-80aa-34e44bf05f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:29.686197 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0d757-de20-40d7-80aa-34e44bf05f5d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:29.686668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0d757-de20-40d7-80aa-34e44bf05f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:29.687133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0d757-de20-40d7-80aa-34e44bf05f5d None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:29.687988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0d757-de20-40d7-80aa-34e44bf05f5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:29.688302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0d757-de20-40d7-80aa-34e44bf05f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:29.689700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0d757-de20-40d7-80aa-34e44bf05f5d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:29.698425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 576/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:53:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:29.704326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812a634-3b5e-4c6b-acd4-fc8918808276 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:29.801664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812a634-3b5e-4c6b-acd4-fc8918808276 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:29.834043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6b831-f2b9-4af5-a501-860f6dd8891a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:29.834905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6b831-f2b9-4af5-a501-860f6dd8891a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:29.835190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6b831-f2b9-4af5-a501-860f6dd8891a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:29.835463 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6b831-f2b9-4af5-a501-860f6dd8891a None admin] PATH_INFO: `/v3/users/5e6518c4f78d41e89e91fd57d8dfe116` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:29.836679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6b831-f2b9-4af5-a501-860f6dd8891a None admin] RBAC: Authorizing `identity:get_user(user_id=5e6518c4f78d41e89e91fd57d8dfe116)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:29.838135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6b831-f2b9-4af5-a501-860f6dd8891a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:29.839739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 575/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:29 2022] GET /identity/v3/users/5e6518c4f78d41e89e91fd57d8dfe116 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:29.845412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0d757-de20-40d7-80aa-34e44bf05f5d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:29.914582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:29.950058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0d757-de20-40d7-80aa-34e44bf05f5d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:29.983177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e501ecf-fb54-411e-aab1-0785d434adbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:29.984026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e501ecf-fb54-411e-aab1-0785d434adbf None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:29.984307 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e501ecf-fb54-411e-aab1-0785d434adbf None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:29.984607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e501ecf-fb54-411e-aab1-0785d434adbf None admin] PATH_INFO: `/v3/projects/34e4b84982ee4f37b77fa6dfe14815bd` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:29.985758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e501ecf-fb54-411e-aab1-0785d434adbf None admin] RBAC: Authorizing `identity:get_project(project_id=34e4b84982ee4f37b77fa6dfe14815bd)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:29.988353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e501ecf-fb54-411e-aab1-0785d434adbf None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:29.990044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 577/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:29 2022] GET /identity/v3/projects/34e4b84982ee4f37b77fa6dfe14815bd => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:29.995884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6b831-f2b9-4af5-a501-860f6dd8891a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:30.040034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:30.095828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6b831-f2b9-4af5-a501-860f6dd8891a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:30.128773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17053105-3f5e-489a-9e80-6f568912a18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:30.129654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053105-3f5e-489a-9e80-6f568912a18c None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:30.129933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053105-3f5e-489a-9e80-6f568912a18c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:30.130203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053105-3f5e-489a-9e80-6f568912a18c None admin] PATH_INFO: `/v3/projects/34e4b84982ee4f37b77fa6dfe14815bd/users/5e6518c4f78d41e89e91fd57d8dfe116/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:30.132127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17053105-3f5e-489a-9e80-6f568912a18c None admin] RBAC: Authorizing `identity:create_grant(project_id=34e4b84982ee4f37b77fa6dfe14815bd, user_id=5e6518c4f78d41e89e91fd57d8dfe116, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:30.133588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17053105-3f5e-489a-9e80-6f568912a18c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:30.146265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 576/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:53:29 2022] PUT /identity/v3/projects/34e4b84982ee4f37b77fa6dfe14815bd/users/5e6518c4f78d41e89e91fd57d8dfe116/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:53:32.394651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faa35c-5418-4a4d-aa1e-b0f1f292dff5 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:32.395209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faa35c-5418-4a4d-aa1e-b0f1f292dff5 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:32.395209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faa35c-5418-4a4d-aa1e-b0f1f292dff5 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:32.396579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 578/1154] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:53:32.401954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824578ea-efc1-45dc-be17-7f8208d78707 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:32.402413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824578ea-efc1-45dc-be17-7f8208d78707 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:32.402515 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824578ea-efc1-45dc-be17-7f8208d78707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:32.419896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-824578ea-efc1-45dc-be17-7f8208d78707 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:32.421125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-824578ea-efc1-45dc-be17-7f8208d78707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:32.486554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 577/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:32.509726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faa35c-5418-4a4d-aa1e-b0f1f292dff5 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:32.608214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faa35c-5418-4a4d-aa1e-b0f1f292dff5 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:32.640506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c38f0-3238-4b89-bec7-1006851abc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:32.641279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c38f0-3238-4b89-bec7-1006851abc71 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:32.641558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c38f0-3238-4b89-bec7-1006851abc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:32.641829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c38f0-3238-4b89-bec7-1006851abc71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:32.645442 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:32.654434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c38f0-3238-4b89-bec7-1006851abc71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-3d3c38f0-3238-4b89-bec7-1006851abc71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:53:32.662974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:32.664097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:53:32.665321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 579/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:53:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:32.670068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824578ea-efc1-45dc-be17-7f8208d78707 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:32.699769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:32.768488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824578ea-efc1-45dc-be17-7f8208d78707 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:32.802335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3444b9-3152-4cee-bb7d-83e6a442214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:32.802997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3444b9-3152-4cee-bb7d-83e6a442214d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:32.803281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3444b9-3152-4cee-bb7d-83e6a442214d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:32.803558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3444b9-3152-4cee-bb7d-83e6a442214d None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:32.804305 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3444b9-3152-4cee-bb7d-83e6a442214d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:32.804648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3444b9-3152-4cee-bb7d-83e6a442214d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:32.806027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3444b9-3152-4cee-bb7d-83e6a442214d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:32.814813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 578/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:53:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:53:32.821766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c38f0-3238-4b89-bec7-1006851abc71 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:32.919845 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c38f0-3238-4b89-bec7-1006851abc71 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:32.953274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85430b2-afd2-462a-a28d-33ee21fb8c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:32.954132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85430b2-afd2-462a-a28d-33ee21fb8c0a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:32.954420 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85430b2-afd2-462a-a28d-33ee21fb8c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:32.954698 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85430b2-afd2-462a-a28d-33ee21fb8c0a None admin] PATH_INFO: `/v3/users/5e6518c4f78d41e89e91fd57d8dfe116` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:32.955976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85430b2-afd2-462a-a28d-33ee21fb8c0a None admin] RBAC: Authorizing `identity:get_user(user_id=5e6518c4f78d41e89e91fd57d8dfe116)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:32.957445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85430b2-afd2-462a-a28d-33ee21fb8c0a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:32.959056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 580/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:32 2022] GET /identity/v3/users/5e6518c4f78d41e89e91fd57d8dfe116 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:32.964788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3444b9-3152-4cee-bb7d-83e6a442214d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:33.055844 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:33.062743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3444b9-3152-4cee-bb7d-83e6a442214d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:33.094912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cc14e-65e2-4a66-81bb-4c1d2c23b0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:33.095549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cc14e-65e2-4a66-81bb-4c1d2c23b0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:33.095831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cc14e-65e2-4a66-81bb-4c1d2c23b0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:33.096106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cc14e-65e2-4a66-81bb-4c1d2c23b0a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:33.161032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cc14e-65e2-4a66-81bb-4c1d2c23b0a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:33.162473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cc14e-65e2-4a66-81bb-4c1d2c23b0a6 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:33.250525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:33.258484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 579/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:53:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:53:33.264205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85430b2-afd2-462a-a28d-33ee21fb8c0a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:33.325017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:33.362680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85430b2-afd2-462a-a28d-33ee21fb8c0a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:33.396752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e054c-4d97-458b-9034-0a49a08e7fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:33.397593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e054c-4d97-458b-9034-0a49a08e7fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:33.397876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e054c-4d97-458b-9034-0a49a08e7fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:33.398152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e054c-4d97-458b-9034-0a49a08e7fc9 None admin] PATH_INFO: `/v3/projects/34e4b84982ee4f37b77fa6dfe14815bd` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:33.399322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e054c-4d97-458b-9034-0a49a08e7fc9 None admin] RBAC: Authorizing `identity:get_project(project_id=34e4b84982ee4f37b77fa6dfe14815bd)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:33.400795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e054c-4d97-458b-9034-0a49a08e7fc9 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:33.402449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 581/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:33 2022] GET /identity/v3/projects/34e4b84982ee4f37b77fa6dfe14815bd => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:33.408333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cc14e-65e2-4a66-81bb-4c1d2c23b0a6 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:33.505472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cc14e-65e2-4a66-81bb-4c1d2c23b0a6 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:33.538897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e3e3a-7776-4d1f-b526-a00e18901311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:33.539542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3e3a-7776-4d1f-b526-a00e18901311 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:33.539824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3e3a-7776-4d1f-b526-a00e18901311 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:33.540094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3e3a-7776-4d1f-b526-a00e18901311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:33.540933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e3e3a-7776-4d1f-b526-a00e18901311 None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=34e4b84982ee4f37b77fa6dfe14815bd, user.id=5e6518c4f78d41e89e91fd57d8dfe116) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:33.541240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e3e3a-7776-4d1f-b526-a00e18901311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:33.542664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e3e3a-7776-4d1f-b526-a00e18901311 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:33.548288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 580/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:53:33 2022] GET /identity/v3/role_assignments?user.id=5e6518c4f78d41e89e91fd57d8dfe116&scope.project.id=34e4b84982ee4f37b77fa6dfe14815bd&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:35.849624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4430c3-15d2-47f5-ba47-35b14d4e88fa None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:35.850199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4430c3-15d2-47f5-ba47-35b14d4e88fa None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:35.850199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4430c3-15d2-47f5-ba47-35b14d4e88fa None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:35.851520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 582/1162] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:35.857231 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a90ef5-d8e0-4c47-9e9c-7e4cec647098 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:35.857650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a90ef5-d8e0-4c47-9e9c-7e4cec647098 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:35.857786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a90ef5-d8e0-4c47-9e9c-7e4cec647098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:35.875672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-39a90ef5-d8e0-4c47-9e9c-7e4cec647098 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:35.877205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-39a90ef5-d8e0-4c47-9e9c-7e4cec647098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:35.940042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:35.956623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 581/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:35.983594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4430c3-15d2-47f5-ba47-35b14d4e88fa None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:36.087011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4430c3-15d2-47f5-ba47-35b14d4e88fa None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:36.116366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:36.121608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7820e00-4186-4e09-b0f0-b752c001a865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:36.122252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820e00-4186-4e09-b0f0-b752c001a865 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:36.122536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820e00-4186-4e09-b0f0-b752c001a865 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:36.122810 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7820e00-4186-4e09-b0f0-b752c001a865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:36.187781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7820e00-4186-4e09-b0f0-b752c001a865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:36.189264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7820e00-4186-4e09-b0f0-b752c001a865 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:36.286828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 583/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:53:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:53:36.292921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a90ef5-d8e0-4c47-9e9c-7e4cec647098 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:36.388250 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a90ef5-d8e0-4c47-9e9c-7e4cec647098 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:36.422178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc72d79-6d93-43fd-8e66-94b166b4a892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:36.422968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc72d79-6d93-43fd-8e66-94b166b4a892 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:36.423250 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc72d79-6d93-43fd-8e66-94b166b4a892 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:36.423522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc72d79-6d93-43fd-8e66-94b166b4a892 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:36.430772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc72d79-6d93-43fd-8e66-94b166b4a892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:36.432206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc72d79-6d93-43fd-8e66-94b166b4a892 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-2cc72d79-6d93-43fd-8e66-94b166b4a892 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:36.446625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:36.447923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:53:36.448871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 582/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jun 8 01:53:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:36.453775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7820e00-4186-4e09-b0f0-b752c001a865 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:36.483291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:36.548926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7820e00-4186-4e09-b0f0-b752c001a865 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:36.582890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee02acc-a93b-402d-8c9d-105a38cfa291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:36.583538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02acc-a93b-402d-8c9d-105a38cfa291 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:36.583819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02acc-a93b-402d-8c9d-105a38cfa291 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:36.584098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee02acc-a93b-402d-8c9d-105a38cfa291 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:36.584833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee02acc-a93b-402d-8c9d-105a38cfa291 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:36.585137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee02acc-a93b-402d-8c9d-105a38cfa291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:36.586490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee02acc-a93b-402d-8c9d-105a38cfa291 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:36.593434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 584/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Jun 8 01:53:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:38.815298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a9263-b90b-4f76-bbce-46030b179586 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:38.815855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a9263-b90b-4f76-bbce-46030b179586 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:38.815855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a9263-b90b-4f76-bbce-46030b179586 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:38.817348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 583/1167] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:38.822634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812f20-c86b-47e9-aeb4-0b900d209e77 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:38.823030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812f20-c86b-47e9-aeb4-0b900d209e77 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:38.823199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812f20-c86b-47e9-aeb4-0b900d209e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:38.840215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-5e812f20-c86b-47e9-aeb4-0b900d209e77 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:38.841428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5e812f20-c86b-47e9-aeb4-0b900d209e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:38.911811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 585/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:38.934270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a9263-b90b-4f76-bbce-46030b179586 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:38.984315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:39.036302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a9263-b90b-4f76-bbce-46030b179586 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:39.068255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0396ab-eac8-4411-8552-94384ab23326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:39.068862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0396ab-eac8-4411-8552-94384ab23326 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:39.069143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0396ab-eac8-4411-8552-94384ab23326 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:39.069415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0396ab-eac8-4411-8552-94384ab23326 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:39.070138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0396ab-eac8-4411-8552-94384ab23326 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:39.071489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0396ab-eac8-4411-8552-94384ab23326 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:39.086338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 584/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Wed Jun 8 01:53:38 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:53:41.349705 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17265-4e0e-4520-ac92-4fa42980fe3e None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:41.350252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17265-4e0e-4520-ac92-4fa42980fe3e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:41.350252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c17265-4e0e-4520-ac92-4fa42980fe3e None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:41.351629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 586/1170] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:41.356765 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafadacf-a221-4c40-a691-941263cbb286 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:41.357137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafadacf-a221-4c40-a691-941263cbb286 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:41.357318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafadacf-a221-4c40-a691-941263cbb286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:41.374420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-cafadacf-a221-4c40-a691-941263cbb286 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:41.375589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cafadacf-a221-4c40-a691-941263cbb286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:41.441079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 585/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:41.465174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c17265-4e0e-4520-ac92-4fa42980fe3e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:41.492411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:41.566604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c17265-4e0e-4520-ac92-4fa42980fe3e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:41.599778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae608f-90ad-4130-8943-0904568cb994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:41.600597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae608f-90ad-4130-8943-0904568cb994 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:41.600884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae608f-90ad-4130-8943-0904568cb994 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:41.601159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae608f-90ad-4130-8943-0904568cb994 None admin] PATH_INFO: `/v3/domains/26fb793dc2574df6a9a21a014e3bba9d` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:41.611249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae608f-90ad-4130-8943-0904568cb994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=26fb793dc2574df6a9a21a014e3bba9d)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:41.612754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae608f-90ad-4130-8943-0904568cb994 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:41.614354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 587/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Jun 8 01:53:41 2022] GET /identity/v3/domains/26fb793dc2574df6a9a21a014e3bba9d => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:41.620052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafadacf-a221-4c40-a691-941263cbb286 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:41.659899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:41.718235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafadacf-a221-4c40-a691-941263cbb286 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:41.751641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50163c8a-6ad5-4cb6-b973-541e7e78b5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:41.752488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50163c8a-6ad5-4cb6-b973-541e7e78b5cc None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:41.752944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50163c8a-6ad5-4cb6-b973-541e7e78b5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:41.753222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50163c8a-6ad5-4cb6-b973-541e7e78b5cc None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:41.753938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50163c8a-6ad5-4cb6-b973-541e7e78b5cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:41.755405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50163c8a-6ad5-4cb6-b973-541e7e78b5cc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:41.780669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 586/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:53:41 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:53:44.044582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b495d73-adad-4b4c-b3be-352717b1e39d None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:44.045135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b495d73-adad-4b4c-b3be-352717b1e39d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:44.045135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b495d73-adad-4b4c-b3be-352717b1e39d None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:44.046490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 588/1174] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:44.052969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284192b-6586-4d7a-a042-660dfc06e073 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:44.053407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284192b-6586-4d7a-a042-660dfc06e073 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:44.053690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284192b-6586-4d7a-a042-660dfc06e073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:44.071853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f284192b-6586-4d7a-a042-660dfc06e073 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:44.073086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f284192b-6586-4d7a-a042-660dfc06e073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:44.137492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 587/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:53:44.163195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b495d73-adad-4b4c-b3be-352717b1e39d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:44.247929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:44.263649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b495d73-adad-4b4c-b3be-352717b1e39d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:44.296253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdba23d-e8ae-4de1-9d3a-ec7b0f662998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:44.297127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdba23d-e8ae-4de1-9d3a-ec7b0f662998 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:44.297405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdba23d-e8ae-4de1-9d3a-ec7b0f662998 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:44.297692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdba23d-e8ae-4de1-9d3a-ec7b0f662998 None admin] PATH_INFO: `/v3/domains/26fb793dc2574df6a9a21a014e3bba9d` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:44.299876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdba23d-e8ae-4de1-9d3a-ec7b0f662998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=26fb793dc2574df6a9a21a014e3bba9d)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:44.301360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdba23d-e8ae-4de1-9d3a-ec7b0f662998 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:44.302932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 589/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Jun 8 01:53:44 2022] GET /identity/v3/domains/26fb793dc2574df6a9a21a014e3bba9d => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:44.308797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284192b-6586-4d7a-a042-660dfc06e073 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:44.329452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:44.406465 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284192b-6586-4d7a-a042-660dfc06e073 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:44.440896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44737579-199a-488b-b1a1-0100ae68d496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:44.441495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737579-199a-488b-b1a1-0100ae68d496 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:44.441774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737579-199a-488b-b1a1-0100ae68d496 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:44.442045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737579-199a-488b-b1a1-0100ae68d496 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:44.442745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44737579-199a-488b-b1a1-0100ae68d496 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:44.444172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44737579-199a-488b-b1a1-0100ae68d496 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:44.464886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 588/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:53:44 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:53:46.738755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec33990-e443-41d9-8d98-cc87b7c624d2 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:46.739317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec33990-e443-41d9-8d98-cc87b7c624d2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:46.739317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec33990-e443-41d9-8d98-cc87b7c624d2 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:46.740691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 590/1178] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:46.745950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb600850-cf7f-40e4-8a97-586d420884a0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:46.746334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb600850-cf7f-40e4-8a97-586d420884a0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:46.746509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb600850-cf7f-40e4-8a97-586d420884a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:46.763390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-cb600850-cf7f-40e4-8a97-586d420884a0 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:46.764608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cb600850-cf7f-40e4-8a97-586d420884a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:46.829563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 589/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:46.852558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec33990-e443-41d9-8d98-cc87b7c624d2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:46.953264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec33990-e443-41d9-8d98-cc87b7c624d2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:46.986401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b071c-e6a8-4035-b909-7e26ffd95505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:46.987180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b071c-e6a8-4035-b909-7e26ffd95505 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:46.987464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b071c-e6a8-4035-b909-7e26ffd95505 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:46.987741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b071c-e6a8-4035-b909-7e26ffd95505 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:46.998006 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b071c-e6a8-4035-b909-7e26ffd95505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c22b071c-e6a8-4035-b909-7e26ffd95505 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:53:47.005535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:47.006675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:53:47.007726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 591/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:53:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:47.012793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb600850-cf7f-40e4-8a97-586d420884a0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:47.021682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:47.109754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb600850-cf7f-40e4-8a97-586d420884a0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:47.142283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc7db9-f9b0-4e14-be9e-2061601f02ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:47.143056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc7db9-f9b0-4e14-be9e-2061601f02ce None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:47.143334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc7db9-f9b0-4e14-be9e-2061601f02ce None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:47.143602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc7db9-f9b0-4e14-be9e-2061601f02ce None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:47.144337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc7db9-f9b0-4e14-be9e-2061601f02ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:47.144670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc7db9-f9b0-4e14-be9e-2061601f02ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:47.146027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc7db9-f9b0-4e14-be9e-2061601f02ce None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:47.153777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 590/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:53:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:47.160376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b071c-e6a8-4035-b909-7e26ffd95505 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:47.176173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:47.261310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b071c-e6a8-4035-b909-7e26ffd95505 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:47.294312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01755188-e6ac-4e4d-ae13-943c834a7d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:47.295172 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01755188-e6ac-4e4d-ae13-943c834a7d0a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:47.295456 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01755188-e6ac-4e4d-ae13-943c834a7d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:47.295731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01755188-e6ac-4e4d-ae13-943c834a7d0a None admin] PATH_INFO: `/v3/users/31c41f3887e2448ab0487d5831a6cbea` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:47.308946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01755188-e6ac-4e4d-ae13-943c834a7d0a None admin] RBAC: Authorizing `identity:get_user(user_id=31c41f3887e2448ab0487d5831a6cbea)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:47.310390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01755188-e6ac-4e4d-ae13-943c834a7d0a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:47.312005 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 592/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:47 2022] GET /identity/v3/users/31c41f3887e2448ab0487d5831a6cbea => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:47.317824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc7db9-f9b0-4e14-be9e-2061601f02ce None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:47.382417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:47.414816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc7db9-f9b0-4e14-be9e-2061601f02ce None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:47.446885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc670-1380-4428-876c-b0b6df3d0ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:47.447513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc670-1380-4428-876c-b0b6df3d0ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:47.447794 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc670-1380-4428-876c-b0b6df3d0ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:47.448067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc670-1380-4428-876c-b0b6df3d0ad4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:47.514544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecc670-1380-4428-876c-b0b6df3d0ad4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:47.515916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecc670-1380-4428-876c-b0b6df3d0ad4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:47.577470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:47.611077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 591/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:53:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:53:47.616956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01755188-e6ac-4e4d-ae13-943c834a7d0a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:47.685070 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:47.714147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01755188-e6ac-4e4d-ae13-943c834a7d0a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:47.746782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b21cb-f2a9-42e0-be84-57bb6d2ebf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:47.747643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b21cb-f2a9-42e0-be84-57bb6d2ebf7b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:47.747921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b21cb-f2a9-42e0-be84-57bb6d2ebf7b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:47.748189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b21cb-f2a9-42e0-be84-57bb6d2ebf7b None admin] PATH_INFO: `/v3/projects/2af96a5aa5e94935ab1f085bf8c76656` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:47.749373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b21cb-f2a9-42e0-be84-57bb6d2ebf7b None admin] RBAC: Authorizing `identity:get_project(project_id=2af96a5aa5e94935ab1f085bf8c76656)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:47.750806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2b21cb-f2a9-42e0-be84-57bb6d2ebf7b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:47.752679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 593/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:47 2022] GET /identity/v3/projects/2af96a5aa5e94935ab1f085bf8c76656 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:47.758715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc670-1380-4428-876c-b0b6df3d0ad4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:47.855783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc670-1380-4428-876c-b0b6df3d0ad4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:47.888805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaddd18-9f03-4da9-8880-78b756a0a4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:47.889447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaddd18-9f03-4da9-8880-78b756a0a4de None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:47.889733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaddd18-9f03-4da9-8880-78b756a0a4de None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:47.890011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaddd18-9f03-4da9-8880-78b756a0a4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:47.890826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaddd18-9f03-4da9-8880-78b756a0a4de None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=2af96a5aa5e94935ab1f085bf8c76656, user.id=31c41f3887e2448ab0487d5831a6cbea) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:47.891138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaddd18-9f03-4da9-8880-78b756a0a4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:47.892855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaddd18-9f03-4da9-8880-78b756a0a4de None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:47.898073 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 592/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:53:47 2022] GET /identity/v3/role_assignments?user.id=31c41f3887e2448ab0487d5831a6cbea&scope.project.id=2af96a5aa5e94935ab1f085bf8c76656&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:50.104060 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da295f-13f5-41e5-9328-ec04a2602097 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:50.104607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da295f-13f5-41e5-9328-ec04a2602097 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:50.104728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da295f-13f5-41e5-9328-ec04a2602097 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:50.105979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 594/1186] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:50.111083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0fc42-dc35-4af4-b293-a4c01ccff215 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:50.111601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0fc42-dc35-4af4-b293-a4c01ccff215 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:50.111926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0fc42-dc35-4af4-b293-a4c01ccff215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:50.129294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5cf0fc42-dc35-4af4-b293-a4c01ccff215 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:50.130486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5cf0fc42-dc35-4af4-b293-a4c01ccff215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:50.156942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:50.198111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 593/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:50.221421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da295f-13f5-41e5-9328-ec04a2602097 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:50.320319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da295f-13f5-41e5-9328-ec04a2602097 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:50.353566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad28f17-4636-4135-bc8e-a2a42583a8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:50.354346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad28f17-4636-4135-bc8e-a2a42583a8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:50.354627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad28f17-4636-4135-bc8e-a2a42583a8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:50.354898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad28f17-4636-4135-bc8e-a2a42583a8f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:50.362437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:50.365730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad28f17-4636-4135-bc8e-a2a42583a8f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-1ad28f17-4636-4135-bc8e-a2a42583a8f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:53:50.373404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:50.374489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:53:50.375528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 595/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:53:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:53:50.381018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0fc42-dc35-4af4-b293-a4c01ccff215 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:50.479200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0fc42-dc35-4af4-b293-a4c01ccff215 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:50.512169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadcfb3-7377-4615-97fc-a08603ffcaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:50.512852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadcfb3-7377-4615-97fc-a08603ffcaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:50.513133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadcfb3-7377-4615-97fc-a08603ffcaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:50.513405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadcfb3-7377-4615-97fc-a08603ffcaa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:50.514135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadcfb3-7377-4615-97fc-a08603ffcaa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:50.514436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadcfb3-7377-4615-97fc-a08603ffcaa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:50.515799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadcfb3-7377-4615-97fc-a08603ffcaa5 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:50.523548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 594/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:53:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:50.529461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad28f17-4636-4135-bc8e-a2a42583a8f8 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:50.628183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad28f17-4636-4135-bc8e-a2a42583a8f8 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:50.660300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67961ed5-1d81-4615-8173-aa6d5973ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:50.661184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67961ed5-1d81-4615-8173-aa6d5973ce5f None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:50.661463 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67961ed5-1d81-4615-8173-aa6d5973ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:50.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67961ed5-1d81-4615-8173-aa6d5973ce5f None admin] PATH_INFO: `/v3/users/31c41f3887e2448ab0487d5831a6cbea` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:50.662891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67961ed5-1d81-4615-8173-aa6d5973ce5f None admin] RBAC: Authorizing `identity:get_user(user_id=31c41f3887e2448ab0487d5831a6cbea)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:50.664320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67961ed5-1d81-4615-8173-aa6d5973ce5f None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:50.665960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 596/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:50 2022] GET /identity/v3/users/31c41f3887e2448ab0487d5831a6cbea => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:50.671755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadcfb3-7377-4615-97fc-a08603ffcaa5 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:50.698895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:50.777592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadcfb3-7377-4615-97fc-a08603ffcaa5 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:50.810938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63458225-e244-4634-a253-caf3e4f79148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:50.811788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63458225-e244-4634-a253-caf3e4f79148 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:50.812066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63458225-e244-4634-a253-caf3e4f79148 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:50.812334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63458225-e244-4634-a253-caf3e4f79148 None admin] PATH_INFO: `/v3/projects/2af96a5aa5e94935ab1f085bf8c76656` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:50.813514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63458225-e244-4634-a253-caf3e4f79148 None admin] RBAC: Authorizing `identity:get_project(project_id=2af96a5aa5e94935ab1f085bf8c76656)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:50.815036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63458225-e244-4634-a253-caf3e4f79148 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:50.816736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 595/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:50 2022] GET /identity/v3/projects/2af96a5aa5e94935ab1f085bf8c76656 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:50.822718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67961ed5-1d81-4615-8173-aa6d5973ce5f None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:50.887444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:50.921199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67961ed5-1d81-4615-8173-aa6d5973ce5f None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:50.953679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02676425-544e-4f05-9bfc-390094cde5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:50.954610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02676425-544e-4f05-9bfc-390094cde5ad None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:50.955053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02676425-544e-4f05-9bfc-390094cde5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:50.955335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02676425-544e-4f05-9bfc-390094cde5ad None admin] PATH_INFO: `/v3/projects/2af96a5aa5e94935ab1f085bf8c76656/users/31c41f3887e2448ab0487d5831a6cbea/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:50.957301 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02676425-544e-4f05-9bfc-390094cde5ad None admin] RBAC: Authorizing `identity:create_grant(project_id=2af96a5aa5e94935ab1f085bf8c76656, user_id=31c41f3887e2448ab0487d5831a6cbea, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:50.958732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02676425-544e-4f05-9bfc-390094cde5ad None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:50.971437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 597/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:53:50 2022] PUT /identity/v3/projects/2af96a5aa5e94935ab1f085bf8c76656/users/31c41f3887e2448ab0487d5831a6cbea/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:53:53.190419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be9d88-b947-4c53-a6ab-9a925c157f07 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:53.190924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be9d88-b947-4c53-a6ab-9a925c157f07 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:53.190924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be9d88-b947-4c53-a6ab-9a925c157f07 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:53.192242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 596/1193] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:53:53.197559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e67b88-7b00-4b14-bd39-afe21719eb93 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:53.197963 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e67b88-7b00-4b14-bd39-afe21719eb93 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:53.198119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e67b88-7b00-4b14-bd39-afe21719eb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:53.214528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-f3e67b88-7b00-4b14-bd39-afe21719eb93 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:53:53.215712 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f3e67b88-7b00-4b14-bd39-afe21719eb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:53:53.280227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 598/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:53:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:53:53.305250 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be9d88-b947-4c53-a6ab-9a925c157f07 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:53.394297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:53.403940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be9d88-b947-4c53-a6ab-9a925c157f07 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:53.436649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6be6e2-5073-4d51-80d6-f4a613c0428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:53.437427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6be6e2-5073-4d51-80d6-f4a613c0428c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:53.437706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6be6e2-5073-4d51-80d6-f4a613c0428c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:53.437976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6be6e2-5073-4d51-80d6-f4a613c0428c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:53.448729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6be6e2-5073-4d51-80d6-f4a613c0428c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-8e6be6e2-5073-4d51-80d6-f4a613c0428c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:53:53.456769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:53:53.457859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:53:53.459115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 597/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:53:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:53:53.463868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e67b88-7b00-4b14-bd39-afe21719eb93 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:53.519908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:53.560716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e67b88-7b00-4b14-bd39-afe21719eb93 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:53.593271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b967639-0492-4923-bb53-32afe16d4bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:53.593926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b967639-0492-4923-bb53-32afe16d4bea None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:53.594211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b967639-0492-4923-bb53-32afe16d4bea None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:53.594487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b967639-0492-4923-bb53-32afe16d4bea None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:53.595362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b967639-0492-4923-bb53-32afe16d4bea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:53.595676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b967639-0492-4923-bb53-32afe16d4bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:53.597072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b967639-0492-4923-bb53-32afe16d4bea None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:53.604868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 599/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:53:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:53.610496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6be6e2-5073-4d51-80d6-f4a613c0428c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:53.709701 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6be6e2-5073-4d51-80d6-f4a613c0428c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:53.742754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb90e7d-7171-4627-97b4-6e09e59812e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:53.743609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90e7d-7171-4627-97b4-6e09e59812e7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:53.743896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90e7d-7171-4627-97b4-6e09e59812e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:53.744174 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90e7d-7171-4627-97b4-6e09e59812e7 None admin] PATH_INFO: `/v3/users/31c41f3887e2448ab0487d5831a6cbea` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:53.745392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb90e7d-7171-4627-97b4-6e09e59812e7 None admin] RBAC: Authorizing `identity:get_user(user_id=31c41f3887e2448ab0487d5831a6cbea)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:53.746831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb90e7d-7171-4627-97b4-6e09e59812e7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:53.748450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 598/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Jun 8 01:53:53 2022] GET /identity/v3/users/31c41f3887e2448ab0487d5831a6cbea => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:53.754724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b967639-0492-4923-bb53-32afe16d4bea None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:53.851322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b967639-0492-4923-bb53-32afe16d4bea None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:53.870214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:53.884632 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec1bd1-71dc-4955-b5c9-10d4acdaf316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:53.885273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec1bd1-71dc-4955-b5c9-10d4acdaf316 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:53.885559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec1bd1-71dc-4955-b5c9-10d4acdaf316 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:53.885840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec1bd1-71dc-4955-b5c9-10d4acdaf316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:53.954001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec1bd1-71dc-4955-b5c9-10d4acdaf316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:53.956178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec1bd1-71dc-4955-b5c9-10d4acdaf316 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:54.052044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 600/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:53:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:53:54.058160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb90e7d-7171-4627-97b4-6e09e59812e7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:54.079664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:54.157320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb90e7d-7171-4627-97b4-6e09e59812e7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:54.190442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a0d88-2d43-4796-b4c3-46f48d1aeb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:54.191296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a0d88-2d43-4796-b4c3-46f48d1aeb3a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:54.191580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a0d88-2d43-4796-b4c3-46f48d1aeb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:54.191858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a0d88-2d43-4796-b4c3-46f48d1aeb3a None admin] PATH_INFO: `/v3/projects/2af96a5aa5e94935ab1f085bf8c76656` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:54.193175 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a0d88-2d43-4796-b4c3-46f48d1aeb3a None admin] RBAC: Authorizing `identity:get_project(project_id=2af96a5aa5e94935ab1f085bf8c76656)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:54.195341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a0d88-2d43-4796-b4c3-46f48d1aeb3a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:54.197404 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 599/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Jun 8 01:53:54 2022] GET /identity/v3/projects/2af96a5aa5e94935ab1f085bf8c76656 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:53:54.203570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec1bd1-71dc-4955-b5c9-10d4acdaf316 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:53:54.224977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:53:54.301104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec1bd1-71dc-4955-b5c9-10d4acdaf316 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:53:54.334025 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56946895-de8e-4a6c-952a-df712731ea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:53:54.334607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56946895-de8e-4a6c-952a-df712731ea7e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:53:54.334890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56946895-de8e-4a6c-952a-df712731ea7e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:53:54.335166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56946895-de8e-4a6c-952a-df712731ea7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:53:54.335980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56946895-de8e-4a6c-952a-df712731ea7e None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=2af96a5aa5e94935ab1f085bf8c76656, user.id=31c41f3887e2448ab0487d5831a6cbea) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:53:54.336288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56946895-de8e-4a6c-952a-df712731ea7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:53:54.337891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56946895-de8e-4a6c-952a-df712731ea7e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:53:54.343522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 601/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:53:54 2022] GET /identity/v3/role_assignments?user.id=31c41f3887e2448ab0487d5831a6cbea&scope.project.id=2af96a5aa5e94935ab1f085bf8c76656&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:54:04.185941 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=56 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:54:04.340993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=58 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:54:55.482186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd6e5-24f2-468f-9961-7941ac53053a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:55.482186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd6e5-24f2-468f-9961-7941ac53053a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:55.483236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd6e5-24f2-468f-9961-7941ac53053a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:55.486900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 600/1201] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:54:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:54:55.491526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22a850-5589-4f2d-8759-e9cf8aa398e0 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:55.492028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22a850-5589-4f2d-8759-e9cf8aa398e0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:55.492127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22a850-5589-4f2d-8759-e9cf8aa398e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:55.511846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-8a22a850-5589-4f2d-8759-e9cf8aa398e0 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:54:55.513623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8a22a850-5589-4f2d-8759-e9cf8aa398e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:54:55.607830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 602/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:54:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:54:55.638846 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dd6e5-24f2-468f-9961-7941ac53053a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:54:55.785335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dd6e5-24f2-468f-9961-7941ac53053a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:54:55.825439 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d959bbf-0e80-4092-8d20-c44a93f215e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:54:55.826239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959bbf-0e80-4092-8d20-c44a93f215e8 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:55.826520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959bbf-0e80-4092-8d20-c44a93f215e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:55.826797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d959bbf-0e80-4092-8d20-c44a93f215e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:55.838202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d959bbf-0e80-4092-8d20-c44a93f215e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:54:55.839723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d959bbf-0e80-4092-8d20-c44a93f215e8 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-1d959bbf-0e80-4092-8d20-c44a93f215e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:54:55.861628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:54:55.862694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:54:55.863685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:54:55.863685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:54:55.863685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:54:55.863685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:54:55.863685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 601/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:54:55.869472 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22a850-5589-4f2d-8759-e9cf8aa398e0 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:54:56.022115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22a850-5589-4f2d-8759-e9cf8aa398e0 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:54:56.057792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:54:56.065243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c02dd3-26f7-496f-8cd0-c7f8ab394ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:54:56.065940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c02dd3-26f7-496f-8cd0-c7f8ab394ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:56.066308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c02dd3-26f7-496f-8cd0-c7f8ab394ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:56.066520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c02dd3-26f7-496f-8cd0-c7f8ab394ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:56.067325 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c02dd3-26f7-496f-8cd0-c7f8ab394ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:54:56.067661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c02dd3-26f7-496f-8cd0-c7f8ab394ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:54:56.069175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c02dd3-26f7-496f-8cd0-c7f8ab394ab1 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:54:56.082434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 603/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:54:56.090171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d959bbf-0e80-4092-8d20-c44a93f215e8 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:54:56.161042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:54:56.224144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d959bbf-0e80-4092-8d20-c44a93f215e8 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:54:56.263810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c9351-3ff4-431d-a6f7-c84904e34932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:54:56.264155 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c9351-3ff4-431d-a6f7-c84904e34932 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:56.264423 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c9351-3ff4-431d-a6f7-c84904e34932 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:56.264768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c9351-3ff4-431d-a6f7-c84904e34932 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:56.265469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c9351-3ff4-431d-a6f7-c84904e34932 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:54:56.267735 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c9351-3ff4-431d-a6f7-c84904e34932 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:54:56.308517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 602/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Jun 8 01:54:56 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:54:58.815544 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef2521-e604-4503-80d6-2b0548350a14 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:58.816153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef2521-e604-4503-80d6-2b0548350a14 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:58.816153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef2521-e604-4503-80d6-2b0548350a14 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:58.817691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 604/1206] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:54:58.822999 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea2696-89d4-48ba-a8cd-d6749e8f80e6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:58.823461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea2696-89d4-48ba-a8cd-d6749e8f80e6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:58.823549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea2696-89d4-48ba-a8cd-d6749e8f80e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:58.841574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a1ea2696-89d4-48ba-a8cd-d6749e8f80e6 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:54:58.842821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a1ea2696-89d4-48ba-a8cd-d6749e8f80e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:54:58.928920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 603/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:54:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:54:58.957077 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef2521-e604-4503-80d6-2b0548350a14 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:54:59.081838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef2521-e604-4503-80d6-2b0548350a14 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:54:59.122997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2521ba7-e988-4638-aeb9-c452e2ec26cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:54:59.123802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2521ba7-e988-4638-aeb9-c452e2ec26cc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:59.124122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2521ba7-e988-4638-aeb9-c452e2ec26cc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:59.124360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2521ba7-e988-4638-aeb9-c452e2ec26cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:59.137574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2521ba7-e988-4638-aeb9-c452e2ec26cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-a2521ba7-e988-4638-aeb9-c452e2ec26cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:54:59.145620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:54:59.146676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:54:59.148144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 605/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:54:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:54:59.153124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea2696-89d4-48ba-a8cd-d6749e8f80e6 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:54:59.200761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:54:59.280983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea2696-89d4-48ba-a8cd-d6749e8f80e6 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:54:59.323009 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09ddea-428e-497e-9be3-7cce0a9c979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:54:59.324385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09ddea-428e-497e-9be3-7cce0a9c979a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:59.324385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09ddea-428e-497e-9be3-7cce0a9c979a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:59.324583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09ddea-428e-497e-9be3-7cce0a9c979a None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:59.325180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09ddea-428e-497e-9be3-7cce0a9c979a None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:54:59.325529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09ddea-428e-497e-9be3-7cce0a9c979a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:54:59.327084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09ddea-428e-497e-9be3-7cce0a9c979a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:54:59.340111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 604/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:54:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:54:59.346952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2521ba7-e988-4638-aeb9-c452e2ec26cc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:54:59.410497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:54:59.468699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2521ba7-e988-4638-aeb9-c452e2ec26cc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:54:59.505593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd684ccc-a02e-4e0a-88dc-9368bb44460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:54:59.506379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd684ccc-a02e-4e0a-88dc-9368bb44460e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:59.506672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd684ccc-a02e-4e0a-88dc-9368bb44460e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:59.506949 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd684ccc-a02e-4e0a-88dc-9368bb44460e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:59.518722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd684ccc-a02e-4e0a-88dc-9368bb44460e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:54:59.521111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd684ccc-a02e-4e0a-88dc-9368bb44460e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-dd684ccc-a02e-4e0a-88dc-9368bb44460e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:54:59.541061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:54:59.544331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:54:59.545285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:54:59.545285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:54:59.545285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:54:59.545285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:54:59.545285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 606/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:54:59.548633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09ddea-428e-497e-9be3-7cce0a9c979a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:54:59.659179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:54:59.662626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09ddea-428e-497e-9be3-7cce0a9c979a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:54:59.703300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dec51e-826c-42f1-904f-f0dedbc7fd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:54:59.703977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dec51e-826c-42f1-904f-f0dedbc7fd64 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:59.704865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dec51e-826c-42f1-904f-f0dedbc7fd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:59.705293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dec51e-826c-42f1-904f-f0dedbc7fd64 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:59.706392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dec51e-826c-42f1-904f-f0dedbc7fd64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:54:59.706778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dec51e-826c-42f1-904f-f0dedbc7fd64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:54:59.708379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dec51e-826c-42f1-904f-f0dedbc7fd64 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:54:59.726799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 605/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:54:59.727975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd684ccc-a02e-4e0a-88dc-9368bb44460e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:54:59.847794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd684ccc-a02e-4e0a-88dc-9368bb44460e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:54:59.849825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:54:59.887481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638c732-54b5-44af-b81f-42fdacfd3263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:54:59.888299 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638c732-54b5-44af-b81f-42fdacfd3263 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:54:59.888655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638c732-54b5-44af-b81f-42fdacfd3263 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:54:59.888967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638c732-54b5-44af-b81f-42fdacfd3263 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:54:59.897376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638c732-54b5-44af-b81f-42fdacfd3263 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:54:59.898926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638c732-54b5-44af-b81f-42fdacfd3263 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-4638c732-54b5-44af-b81f-42fdacfd3263 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:54:59.916337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:54:59.918198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:54:59.919453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:54:59.919453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:54:59.919453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:54:59.919453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:54:59.919453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:54:59.919453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:54:59.919453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:54:59.919453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:54:59.919453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:54:59.919453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:54:59.921724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 607/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:54:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:54:59.923213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dec51e-826c-42f1-904f-f0dedbc7fd64 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:00.050471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dec51e-826c-42f1-904f-f0dedbc7fd64 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:00.094109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9aedc-e102-4121-8f6a-23b60e736553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:00.095467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9aedc-e102-4121-8f6a-23b60e736553 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:00.095871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9aedc-e102-4121-8f6a-23b60e736553 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:00.096826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9aedc-e102-4121-8f6a-23b60e736553 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:00.097203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9aedc-e102-4121-8f6a-23b60e736553 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:00.097506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9aedc-e102-4121-8f6a-23b60e736553 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:00.099296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9aedc-e102-4121-8f6a-23b60e736553 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:00.121889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 606/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jun 8 01:54:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:00.128793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638c732-54b5-44af-b81f-42fdacfd3263 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:00.260422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638c732-54b5-44af-b81f-42fdacfd3263 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:00.312905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94911eb-9224-4fec-9dd1-b183d1bc529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:00.313648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94911eb-9224-4fec-9dd1-b183d1bc529c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:00.313964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94911eb-9224-4fec-9dd1-b183d1bc529c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:00.314277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94911eb-9224-4fec-9dd1-b183d1bc529c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:00.354199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:00.413213 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94911eb-9224-4fec-9dd1-b183d1bc529c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:00.414954 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94911eb-9224-4fec-9dd1-b183d1bc529c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:00.542306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 608/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:55:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:55:00.547766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9aedc-e102-4121-8f6a-23b60e736553 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:00.570712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:00.662422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9aedc-e102-4121-8f6a-23b60e736553 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:00.700900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cdb6c-b42a-4540-94f3-cbe7640a23fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:00.701713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cdb6c-b42a-4540-94f3-cbe7640a23fe None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:00.701973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cdb6c-b42a-4540-94f3-cbe7640a23fe None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:00.702236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cdb6c-b42a-4540-94f3-cbe7640a23fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:00.713672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cdb6c-b42a-4540-94f3-cbe7640a23fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:00.715146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cdb6c-b42a-4540-94f3-cbe7640a23fe None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-e82cdb6c-b42a-4540-94f3-cbe7640a23fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:00.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:00.735349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:00.736835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:00.736835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:00.736835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:00.736835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:00.736835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 607/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:00.739877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94911eb-9224-4fec-9dd1-b183d1bc529c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:00.810030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:00.857884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94911eb-9224-4fec-9dd1-b183d1bc529c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:00.896416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c80cc-851c-4aa5-a95c-358d288a31c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:00.897145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c80cc-851c-4aa5-a95c-358d288a31c6 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:00.898030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c80cc-851c-4aa5-a95c-358d288a31c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:00.898030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c80cc-851c-4aa5-a95c-358d288a31c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:00.898464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c80cc-851c-4aa5-a95c-358d288a31c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:00.898778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c80cc-851c-4aa5-a95c-358d288a31c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:00.900162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c80cc-851c-4aa5-a95c-358d288a31c6 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:00.919001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cdb6c-b42a-4540-94f3-cbe7640a23fe None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:00.919448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 609/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:01.003994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:01.041859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cdb6c-b42a-4540-94f3-cbe7640a23fe None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:01.083711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818ced0-b3d3-4df5-96f5-c32d322db75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:01.083711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818ced0-b3d3-4df5-96f5-c32d322db75d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:01.083711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818ced0-b3d3-4df5-96f5-c32d322db75d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:01.083711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818ced0-b3d3-4df5-96f5-c32d322db75d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:01.091479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818ced0-b3d3-4df5-96f5-c32d322db75d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:01.093413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818ced0-b3d3-4df5-96f5-c32d322db75d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a818ced0-b3d3-4df5-96f5-c32d322db75d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:55:01.111412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:01.112651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:01.113864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:55:01.113864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:55:01.113864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:55:01.113864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:55:01.113864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:55:01.113864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:55:01.113864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:01.113864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:01.121007 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 608/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:55:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:01.122187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c80cc-851c-4aa5-a95c-358d288a31c6 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:01.241131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c80cc-851c-4aa5-a95c-358d288a31c6 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:01.267440 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:01.281593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb3811-026b-4206-9814-2a4b179172de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:01.282292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb3811-026b-4206-9814-2a4b179172de None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:01.282615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb3811-026b-4206-9814-2a4b179172de None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:01.282890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb3811-026b-4206-9814-2a4b179172de None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:01.283678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb3811-026b-4206-9814-2a4b179172de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:01.283984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb3811-026b-4206-9814-2a4b179172de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:01.285741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb3811-026b-4206-9814-2a4b179172de None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:01.300949 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 610/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:55:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:01.306964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818ced0-b3d3-4df5-96f5-c32d322db75d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:01.421910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818ced0-b3d3-4df5-96f5-c32d322db75d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:01.458516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:01.465577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab93f8-6deb-45dc-bcb0-26c0b9fd9eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:01.466394 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab93f8-6deb-45dc-bcb0-26c0b9fd9eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:01.466776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab93f8-6deb-45dc-bcb0-26c0b9fd9eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:01.467163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab93f8-6deb-45dc-bcb0-26c0b9fd9eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:01.468205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab93f8-6deb-45dc-bcb0-26c0b9fd9eb9 None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=4eb69af145154c6d85bd4b845979e6c0) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:01.468556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab93f8-6deb-45dc-bcb0-26c0b9fd9eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:01.470604 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab93f8-6deb-45dc-bcb0-26c0b9fd9eb9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:01.477110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 609/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:55:01 2022] GET /identity/v3/role_assignments?user.id=4eb69af145154c6d85bd4b845979e6c0&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:03.948336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af2be-d1fc-42bd-b044-b81c8cd12b17 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:03.949439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af2be-d1fc-42bd-b044-b81c8cd12b17 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:03.949959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af2be-d1fc-42bd-b044-b81c8cd12b17 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:03.951698 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 611/1220] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:55:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:55:03.959309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ce94f-483a-4a0f-a611-5e72bdf7a82f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:03.959803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ce94f-483a-4a0f-a611-5e72bdf7a82f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:03.960016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ce94f-483a-4a0f-a611-5e72bdf7a82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:03.982858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a62ce94f-483a-4a0f-a611-5e72bdf7a82f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:55:03.984485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a62ce94f-483a-4a0f-a611-5e72bdf7a82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:55:04.062386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 610/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:55:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:55:04.086591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11af2be-d1fc-42bd-b044-b81c8cd12b17 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:04.197425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11af2be-d1fc-42bd-b044-b81c8cd12b17 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:04.234329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cb57d-4c8f-41be-924f-8bda6e08724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:04.235127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cb57d-4c8f-41be-924f-8bda6e08724c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:04.235413 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cb57d-4c8f-41be-924f-8bda6e08724c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:04.236329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cb57d-4c8f-41be-924f-8bda6e08724c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:04.247016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cb57d-4c8f-41be-924f-8bda6e08724c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-e70cb57d-4c8f-41be-924f-8bda6e08724c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:55:04.254862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:04.255939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:04.261942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 612/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:55:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:04.262702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ce94f-483a-4a0f-a611-5e72bdf7a82f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:04.384568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ce94f-483a-4a0f-a611-5e72bdf7a82f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:04.413673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:04.430798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e86b5-9a27-4d6a-8c83-80376a690a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:04.431902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e86b5-9a27-4d6a-8c83-80376a690a46 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:04.432377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e86b5-9a27-4d6a-8c83-80376a690a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:04.433165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e86b5-9a27-4d6a-8c83-80376a690a46 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:04.434516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e86b5-9a27-4d6a-8c83-80376a690a46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:04.435177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e86b5-9a27-4d6a-8c83-80376a690a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:04.437053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e86b5-9a27-4d6a-8c83-80376a690a46 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:04.449716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 611/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:55:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:04.456018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cb57d-4c8f-41be-924f-8bda6e08724c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:04.507326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:04.576868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cb57d-4c8f-41be-924f-8bda6e08724c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:04.621618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612ab6b-9411-4c16-9cc2-dead4f3a19b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:04.622510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ab6b-9411-4c16-9cc2-dead4f3a19b7 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:04.622792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ab6b-9411-4c16-9cc2-dead4f3a19b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:04.623106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612ab6b-9411-4c16-9cc2-dead4f3a19b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:04.635181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612ab6b-9411-4c16-9cc2-dead4f3a19b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:04.636767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612ab6b-9411-4c16-9cc2-dead4f3a19b7 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c612ab6b-9411-4c16-9cc2-dead4f3a19b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:04.654650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:04.655668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:04.656730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:04.656730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:04.656730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:04.656730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:04.658858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 613/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:04.665131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e86b5-9a27-4d6a-8c83-80376a690a46 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:04.781259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e86b5-9a27-4d6a-8c83-80376a690a46 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:04.821063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d2a3f-c6c5-4f9e-9998-23acb5dd1b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:04.821819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d2a3f-c6c5-4f9e-9998-23acb5dd1b6c None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:04.822127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d2a3f-c6c5-4f9e-9998-23acb5dd1b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:04.822384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d2a3f-c6c5-4f9e-9998-23acb5dd1b6c None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:04.823191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d2a3f-c6c5-4f9e-9998-23acb5dd1b6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:04.823597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d2a3f-c6c5-4f9e-9998-23acb5dd1b6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:04.825123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d2a3f-c6c5-4f9e-9998-23acb5dd1b6c None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:04.838164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 612/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:55:04.844101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612ab6b-9411-4c16-9cc2-dead4f3a19b7 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:04.952111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:04.959984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612ab6b-9411-4c16-9cc2-dead4f3a19b7 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:04.995470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da64a8-ee2b-458a-aea2-589fda5cdc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:04.996276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da64a8-ee2b-458a-aea2-589fda5cdc7e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:04.996590 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da64a8-ee2b-458a-aea2-589fda5cdc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:04.996870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da64a8-ee2b-458a-aea2-589fda5cdc7e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:05.004961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da64a8-ee2b-458a-aea2-589fda5cdc7e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:05.006420 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da64a8-ee2b-458a-aea2-589fda5cdc7e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c5da64a8-ee2b-458a-aea2-589fda5cdc7e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:55:05.023056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:05.024410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:05.025609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 614/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:05.030233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d2a3f-c6c5-4f9e-9998-23acb5dd1b6c None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:05.080543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:05.139069 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d2a3f-c6c5-4f9e-9998-23acb5dd1b6c None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:05.179116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41376a5-6d76-4624-b69e-db74ea276353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:05.180038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41376a5-6d76-4624-b69e-db74ea276353 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:05.180322 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41376a5-6d76-4624-b69e-db74ea276353 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:05.180865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41376a5-6d76-4624-b69e-db74ea276353 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:05.182527 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41376a5-6d76-4624-b69e-db74ea276353 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:05.183022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41376a5-6d76-4624-b69e-db74ea276353 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:05.185189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41376a5-6d76-4624-b69e-db74ea276353 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:05.209848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 613/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jun 8 01:55:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:05.215732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da64a8-ee2b-458a-aea2-589fda5cdc7e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:05.329230 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da64a8-ee2b-458a-aea2-589fda5cdc7e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:05.368686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bc5b2-464f-4142-bbd7-fa23c5f9a1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:05.369404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bc5b2-464f-4142-bbd7-fa23c5f9a1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:05.369720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bc5b2-464f-4142-bbd7-fa23c5f9a1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:05.369957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bc5b2-464f-4142-bbd7-fa23c5f9a1c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:05.382240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bc5b2-464f-4142-bbd7-fa23c5f9a1c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:05.383716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963bc5b2-464f-4142-bbd7-fa23c5f9a1c4 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-963bc5b2-464f-4142-bbd7-fa23c5f9a1c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:05.401381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:05.402374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:05.403273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:05.403273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:05.403273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:05.403273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:05.403273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 615/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:05.408744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41376a5-6d76-4624-b69e-db74ea276353 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:05.521044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41376a5-6d76-4624-b69e-db74ea276353 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:05.523387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:05.558174 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd86d4a-d57f-439e-ad97-77796f9eef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:05.558835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd86d4a-d57f-439e-ad97-77796f9eef97 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:05.559114 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd86d4a-d57f-439e-ad97-77796f9eef97 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:05.559383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd86d4a-d57f-439e-ad97-77796f9eef97 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:05.560087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd86d4a-d57f-439e-ad97-77796f9eef97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:05.560387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd86d4a-d57f-439e-ad97-77796f9eef97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:05.561898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd86d4a-d57f-439e-ad97-77796f9eef97 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:05.574183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 614/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:05.580827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bc5b2-464f-4142-bbd7-fa23c5f9a1c4 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:05.590216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:05.700572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bc5b2-464f-4142-bbd7-fa23c5f9a1c4 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:05.747210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b512b2-e11d-4ba4-9fc4-6c76018e00aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:05.748240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b512b2-e11d-4ba4-9fc4-6c76018e00aa None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:05.748756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b512b2-e11d-4ba4-9fc4-6c76018e00aa None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:05.749189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b512b2-e11d-4ba4-9fc4-6c76018e00aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:05.756982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b512b2-e11d-4ba4-9fc4-6c76018e00aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:05.758438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b512b2-e11d-4ba4-9fc4-6c76018e00aa None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-e5b512b2-e11d-4ba4-9fc4-6c76018e00aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:55:05.774880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:05.776125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:05.777391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:55:05.777391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:55:05.777391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:55:05.777391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:55:05.777391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:55:05.777391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:55:05.777391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:05.777391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:05.777391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 616/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:55:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:05.783947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd86d4a-d57f-439e-ad97-77796f9eef97 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:05.899141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd86d4a-d57f-439e-ad97-77796f9eef97 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:05.943644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd36c93-37f4-4a2c-9f49-c943ae0bba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:05.944310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd36c93-37f4-4a2c-9f49-c943ae0bba33 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:05.944627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd36c93-37f4-4a2c-9f49-c943ae0bba33 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:05.944904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd36c93-37f4-4a2c-9f49-c943ae0bba33 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:05.945726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd36c93-37f4-4a2c-9f49-c943ae0bba33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:05.946060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd36c93-37f4-4a2c-9f49-c943ae0bba33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:05.947444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd36c93-37f4-4a2c-9f49-c943ae0bba33 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:05.965854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 615/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:55:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:05.967055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b512b2-e11d-4ba4-9fc4-6c76018e00aa None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:06.044340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:06.083501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b512b2-e11d-4ba4-9fc4-6c76018e00aa None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:06.119773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af51f6-086a-4407-9cd7-da87922f6187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:06.120708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af51f6-086a-4407-9cd7-da87922f6187 None admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:06.121031 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af51f6-086a-4407-9cd7-da87922f6187 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:06.121304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af51f6-086a-4407-9cd7-da87922f6187 None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/4eb69af145154c6d85bd4b845979e6c0/roles/32b18412b6c944b8899ef24b902885d2` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:06.138201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af51f6-086a-4407-9cd7-da87922f6187 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=4eb69af145154c6d85bd4b845979e6c0, role_id=32b18412b6c944b8899ef24b902885d2)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:06.139718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af51f6-086a-4407-9cd7-da87922f6187 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:06.153490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 617/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:55:05 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/4eb69af145154c6d85bd4b845979e6c0/roles/32b18412b6c944b8899ef24b902885d2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 01:55:08.510379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5bae-af57-46af-a288-3d9b61e6f266 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:08.510928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5bae-af57-46af-a288-3d9b61e6f266 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:08.511022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5bae-af57-46af-a288-3d9b61e6f266 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:08.512556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 616/1233] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:55:08.518763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ad3f1-95da-44d7-97ae-409a7fa0976b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:08.519191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ad3f1-95da-44d7-97ae-409a7fa0976b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:08.519508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ad3f1-95da-44d7-97ae-409a7fa0976b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:08.536810 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-505ad3f1-95da-44d7-97ae-409a7fa0976b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:55:08.538019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-505ad3f1-95da-44d7-97ae-409a7fa0976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:55:08.609781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 618/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:55:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:55:08.636339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d5bae-af57-46af-a288-3d9b61e6f266 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:08.667862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:08.762422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d5bae-af57-46af-a288-3d9b61e6f266 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:08.799052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e68eb-1e90-41d3-a773-b30a622c5bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:08.799893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e68eb-1e90-41d3-a773-b30a622c5bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:08.800156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e68eb-1e90-41d3-a773-b30a622c5bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:08.800477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e68eb-1e90-41d3-a773-b30a622c5bb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:08.811509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e68eb-1e90-41d3-a773-b30a622c5bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-602e68eb-1e90-41d3-a773-b30a622c5bb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:55:08.819625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:08.820728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:08.821570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 617/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Jun 8 01:55:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:08.827015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ad3f1-95da-44d7-97ae-409a7fa0976b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:08.882511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:08.949645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ad3f1-95da-44d7-97ae-409a7fa0976b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:08.988924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cd991-624e-4791-bba3-4838c3baf0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:08.989654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd991-624e-4791-bba3-4838c3baf0da None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:08.990104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd991-624e-4791-bba3-4838c3baf0da None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:08.990384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cd991-624e-4791-bba3-4838c3baf0da None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:08.991213 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cd991-624e-4791-bba3-4838c3baf0da None admin] RBAC: Adding query filter params (name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:08.991518 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cd991-624e-4791-bba3-4838c3baf0da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:08.993340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cd991-624e-4791-bba3-4838c3baf0da None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:09.002732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 619/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Jun 8 01:55:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:09.008627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e68eb-1e90-41d3-a773-b30a622c5bb1 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:09.107998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:09.126918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e68eb-1e90-41d3-a773-b30a622c5bb1 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:09.167499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe07779-0135-451a-9e66-a5bd82d8452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:09.168510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe07779-0135-451a-9e66-a5bd82d8452a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:09.168801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe07779-0135-451a-9e66-a5bd82d8452a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:09.169075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe07779-0135-451a-9e66-a5bd82d8452a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:09.181661 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe07779-0135-451a-9e66-a5bd82d8452a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:09.183131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe07779-0135-451a-9e66-a5bd82d8452a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-9fe07779-0135-451a-9e66-a5bd82d8452a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:09.203821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:09.205672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:09.206788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:09.206788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:09.206788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:09.206788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:09.206788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 618/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:55:09.212096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cd991-624e-4791-bba3-4838c3baf0da None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:09.327541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cd991-624e-4791-bba3-4838c3baf0da None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:09.338364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:09.365431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b321a6-4fd4-49e0-9ac6-0326c5113e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:09.366026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b321a6-4fd4-49e0-9ac6-0326c5113e67 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:09.366311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b321a6-4fd4-49e0-9ac6-0326c5113e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:09.366587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b321a6-4fd4-49e0-9ac6-0326c5113e67 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:09.367312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b321a6-4fd4-49e0-9ac6-0326c5113e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:09.367619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b321a6-4fd4-49e0-9ac6-0326c5113e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:09.369029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b321a6-4fd4-49e0-9ac6-0326c5113e67 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:09.381031 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 620/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:55:09.386979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe07779-0135-451a-9e66-a5bd82d8452a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:09.498395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe07779-0135-451a-9e66-a5bd82d8452a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:09.534555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085effac-8f29-4ca6-b440-c5da73c3cecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:09.535617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085effac-8f29-4ca6-b440-c5da73c3cecd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:09.535898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085effac-8f29-4ca6-b440-c5da73c3cecd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:09.536171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085effac-8f29-4ca6-b440-c5da73c3cecd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:09.539055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:09.544652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085effac-8f29-4ca6-b440-c5da73c3cecd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:09.546307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085effac-8f29-4ca6-b440-c5da73c3cecd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-085effac-8f29-4ca6-b440-c5da73c3cecd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:55:09.561531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:09.562696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:09.563912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 619/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:09.568881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b321a6-4fd4-49e0-9ac6-0326c5113e67 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:09.681591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b321a6-4fd4-49e0-9ac6-0326c5113e67 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:09.725759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f5536-b3d2-4272-b010-b8c038dc9a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:09.726485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f5536-b3d2-4272-b010-b8c038dc9a4a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:09.726772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f5536-b3d2-4272-b010-b8c038dc9a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:09.727046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f5536-b3d2-4272-b010-b8c038dc9a4a None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:09.727894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f5536-b3d2-4272-b010-b8c038dc9a4a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:09.728202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f5536-b3d2-4272-b010-b8c038dc9a4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:09.729689 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f5536-b3d2-4272-b010-b8c038dc9a4a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:09.753372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 621/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jun 8 01:55:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:09.754930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085effac-8f29-4ca6-b440-c5da73c3cecd None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:09.866465 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085effac-8f29-4ca6-b440-c5da73c3cecd None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:09.909471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3ecf4-03c4-4131-a426-730ba362d4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:09.910143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3ecf4-03c4-4131-a426-730ba362d4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:09.910682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3ecf4-03c4-4131-a426-730ba362d4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:09.911010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3ecf4-03c4-4131-a426-730ba362d4b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:09.982582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:09.991544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3ecf4-03c4-4131-a426-730ba362d4b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:09.993253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3ecf4-03c4-4131-a426-730ba362d4b9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:10.114492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 620/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:55:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 08 01:55:10.120189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f5536-b3d2-4272-b010-b8c038dc9a4a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:10.148514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:10.227336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f5536-b3d2-4272-b010-b8c038dc9a4a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:10.270457 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123abae-5c53-4b3f-8418-f152f09179a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:10.271453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123abae-5c53-4b3f-8418-f152f09179a0 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:10.271735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123abae-5c53-4b3f-8418-f152f09179a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:10.272011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123abae-5c53-4b3f-8418-f152f09179a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:10.284747 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123abae-5c53-4b3f-8418-f152f09179a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:10.286452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123abae-5c53-4b3f-8418-f152f09179a0 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-b123abae-5c53-4b3f-8418-f152f09179a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:10.308267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:10.309237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:10.310165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:10.310165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:10.310165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:10.310165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:10.310165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 622/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:10.316015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3ecf4-03c4-4131-a426-730ba362d4b9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:10.432507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:10.440201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3ecf4-03c4-4131-a426-730ba362d4b9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:10.477696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ac825-2006-497e-a783-9b6debfb53c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:10.478410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ac825-2006-497e-a783-9b6debfb53c4 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:10.478820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ac825-2006-497e-a783-9b6debfb53c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:10.479136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ac825-2006-497e-a783-9b6debfb53c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:10.480072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ac825-2006-497e-a783-9b6debfb53c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:10.480446 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ac825-2006-497e-a783-9b6debfb53c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:10.481945 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ac825-2006-497e-a783-9b6debfb53c4 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:10.500264 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 621/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:10.504565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123abae-5c53-4b3f-8418-f152f09179a0 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:10.592065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:10.617762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123abae-5c53-4b3f-8418-f152f09179a0 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:10.662125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8a2f3-8301-4741-977b-eec5bef3c4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:10.663021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8a2f3-8301-4741-977b-eec5bef3c4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:10.663415 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8a2f3-8301-4741-977b-eec5bef3c4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:10.663842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8a2f3-8301-4741-977b-eec5bef3c4b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:10.677940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8a2f3-8301-4741-977b-eec5bef3c4b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:10.680310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8a2f3-8301-4741-977b-eec5bef3c4b3 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-53e8a2f3-8301-4741-977b-eec5bef3c4b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:55:10.700143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:10.701177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:10.702083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:55:10.702083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:55:10.702083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:55:10.702083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:55:10.702083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:55:10.702083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:55:10.702083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:10.702083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:10.702083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 623/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:10.711121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ac825-2006-497e-a783-9b6debfb53c4 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:10.828288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ac825-2006-497e-a783-9b6debfb53c4 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:10.863534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1af2d6-95b1-4403-a5aa-fc791fbac121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:10.864118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1af2d6-95b1-4403-a5aa-fc791fbac121 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:10.864422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1af2d6-95b1-4403-a5aa-fc791fbac121 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:10.864704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1af2d6-95b1-4403-a5aa-fc791fbac121 None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:10.865492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1af2d6-95b1-4403-a5aa-fc791fbac121 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:10.865856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1af2d6-95b1-4403-a5aa-fc791fbac121 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:10.867232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1af2d6-95b1-4403-a5aa-fc791fbac121 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:10.880376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 622/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:55:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:10.886459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8a2f3-8301-4741-977b-eec5bef3c4b3 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:11.002937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8a2f3-8301-4741-977b-eec5bef3c4b3 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:11.043098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48347b66-aed1-46c2-9836-2ce958310fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:11.043921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48347b66-aed1-46c2-9836-2ce958310fee None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:11.044212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48347b66-aed1-46c2-9836-2ce958310fee None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:11.044657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48347b66-aed1-46c2-9836-2ce958310fee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:11.045512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48347b66-aed1-46c2-9836-2ce958310fee None admin] RBAC: Adding query filter params (role.id=32b18412b6c944b8899ef24b902885d2, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=4eb69af145154c6d85bd4b845979e6c0) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:11.045831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48347b66-aed1-46c2-9836-2ce958310fee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:11.047437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48347b66-aed1-46c2-9836-2ce958310fee None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:11.055617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 624/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:55:10 2022] GET /identity/v3/role_assignments?user.id=4eb69af145154c6d85bd4b845979e6c0&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=32b18412b6c944b8899ef24b902885d2 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:13.605998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88140b3-c83a-41f7-a948-ccd41c8e1cf9 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:13.606577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88140b3-c83a-41f7-a948-ccd41c8e1cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:13.606577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88140b3-c83a-41f7-a948-ccd41c8e1cf9 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:13.608070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 623/1247] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:55:13.616103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb09f2-8528-4d87-8f5e-f55182fd548c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:13.616726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb09f2-8528-4d87-8f5e-f55182fd548c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:13.617160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb09f2-8528-4d87-8f5e-f55182fd548c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:13.624964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:13.642670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b1eb09f2-8528-4d87-8f5e-f55182fd548c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:55:13.644353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b1eb09f2-8528-4d87-8f5e-f55182fd548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:55:13.724368 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 625/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:55:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:55:13.753208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88140b3-c83a-41f7-a948-ccd41c8e1cf9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:13.776901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:13.869067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88140b3-c83a-41f7-a948-ccd41c8e1cf9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:13.906502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9fe27-dcf0-454f-ae83-646f511460ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:13.907938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9fe27-dcf0-454f-ae83-646f511460ac None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:13.908279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9fe27-dcf0-454f-ae83-646f511460ac None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:13.911354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9fe27-dcf0-454f-ae83-646f511460ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:13.922165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9fe27-dcf0-454f-ae83-646f511460ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-e0f9fe27-dcf0-454f-ae83-646f511460ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:55:13.930066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:13.931202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:13.932273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 624/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:55:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:55:13.939677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb09f2-8528-4d87-8f5e-f55182fd548c None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:14.048760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb09f2-8528-4d87-8f5e-f55182fd548c None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:14.080264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:14.085924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c21be7-5603-495d-aeb4-742463d99eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:14.086610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21be7-5603-495d-aeb4-742463d99eac None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:14.087088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21be7-5603-495d-aeb4-742463d99eac None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:14.087535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21be7-5603-495d-aeb4-742463d99eac None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:14.088300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c21be7-5603-495d-aeb4-742463d99eac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:14.088855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c21be7-5603-495d-aeb4-742463d99eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:14.090255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c21be7-5603-495d-aeb4-742463d99eac None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:14.099073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 626/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:55:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:14.104779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9fe27-dcf0-454f-ae83-646f511460ac None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:14.185651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:14.213001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9fe27-dcf0-454f-ae83-646f511460ac None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:14.251824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472e591-48b1-45d4-b74f-3b5954e202c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:14.252387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472e591-48b1-45d4-b74f-3b5954e202c7 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:14.252692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472e591-48b1-45d4-b74f-3b5954e202c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:14.252967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472e591-48b1-45d4-b74f-3b5954e202c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:14.264111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472e591-48b1-45d4-b74f-3b5954e202c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:14.265621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472e591-48b1-45d4-b74f-3b5954e202c7 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-3472e591-48b1-45d4-b74f-3b5954e202c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:14.285300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:14.286974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:14.288628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:14.288628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:14.288628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:14.288628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:14.291653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 625/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:14.292702 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c21be7-5603-495d-aeb4-742463d99eac None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:14.408140 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c21be7-5603-495d-aeb4-742463d99eac None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:14.447033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16baf25-8ca4-4934-ae28-7695e07501e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:14.447635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16baf25-8ca4-4934-ae28-7695e07501e2 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:14.447917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16baf25-8ca4-4934-ae28-7695e07501e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:14.448196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16baf25-8ca4-4934-ae28-7695e07501e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:14.449158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16baf25-8ca4-4934-ae28-7695e07501e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:14.449633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16baf25-8ca4-4934-ae28-7695e07501e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:14.451809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16baf25-8ca4-4934-ae28-7695e07501e2 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:14.466925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 627/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:55:14.473165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472e591-48b1-45d4-b74f-3b5954e202c7 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:14.586159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472e591-48b1-45d4-b74f-3b5954e202c7 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:14.613564 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:14.623321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf0fb8-7edf-4613-95a8-efb33fb0a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:14.624329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf0fb8-7edf-4613-95a8-efb33fb0a68a None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:14.624774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf0fb8-7edf-4613-95a8-efb33fb0a68a None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:14.625172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf0fb8-7edf-4613-95a8-efb33fb0a68a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:14.633580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf0fb8-7edf-4613-95a8-efb33fb0a68a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:14.635275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf0fb8-7edf-4613-95a8-efb33fb0a68a None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-1faf0fb8-7edf-4613-95a8-efb33fb0a68a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:55:14.653512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:14.655339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:14.658159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 626/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:14.663095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16baf25-8ca4-4934-ae28-7695e07501e2 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:14.727041 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:14.773829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16baf25-8ca4-4934-ae28-7695e07501e2 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:14.813136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715520a-aec5-4adf-8b46-42981af48f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:14.814607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715520a-aec5-4adf-8b46-42981af48f52 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:14.814929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715520a-aec5-4adf-8b46-42981af48f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:14.815218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715520a-aec5-4adf-8b46-42981af48f52 None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:14.816533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715520a-aec5-4adf-8b46-42981af48f52 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:14.816821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715520a-aec5-4adf-8b46-42981af48f52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:14.818691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715520a-aec5-4adf-8b46-42981af48f52 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:14.839469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 628/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jun 8 01:55:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:55:14.846321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf0fb8-7edf-4613-95a8-efb33fb0a68a None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:14.957762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf0fb8-7edf-4613-95a8-efb33fb0a68a None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:14.993866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90404746-fd7d-4960-b57d-cd2d27e8cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:14.994452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90404746-fd7d-4960-b57d-cd2d27e8cb9f None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:14.994737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90404746-fd7d-4960-b57d-cd2d27e8cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:14.995025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90404746-fd7d-4960-b57d-cd2d27e8cb9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:15.054600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:15.068117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90404746-fd7d-4960-b57d-cd2d27e8cb9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:15.069657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90404746-fd7d-4960-b57d-cd2d27e8cb9f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:15.187483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 627/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:55:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:55:15.188915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715520a-aec5-4adf-8b46-42981af48f52 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:15.301111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715520a-aec5-4adf-8b46-42981af48f52 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:15.317220 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:15.341777 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc3a09-9b22-428f-a934-84057638017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:15.342722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc3a09-9b22-428f-a934-84057638017b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:15.343106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc3a09-9b22-428f-a934-84057638017b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:15.343391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc3a09-9b22-428f-a934-84057638017b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:15.354918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc3a09-9b22-428f-a934-84057638017b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:15.356457 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc3a09-9b22-428f-a934-84057638017b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-a7dc3a09-9b22-428f-a934-84057638017b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:15.374329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:15.376617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:15.377722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:15.377722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:15.377722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:15.377722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:15.377722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 629/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:15.383482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90404746-fd7d-4960-b57d-cd2d27e8cb9f None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:15.509528 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:15.525404 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90404746-fd7d-4960-b57d-cd2d27e8cb9f None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:15.565829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558ac7c-bdf6-4b71-88a0-f68259188b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:15.566447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558ac7c-bdf6-4b71-88a0-f68259188b89 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:15.566736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558ac7c-bdf6-4b71-88a0-f68259188b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:15.567016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558ac7c-bdf6-4b71-88a0-f68259188b89 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:15.567757 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558ac7c-bdf6-4b71-88a0-f68259188b89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:15.568065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558ac7c-bdf6-4b71-88a0-f68259188b89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:15.569521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558ac7c-bdf6-4b71-88a0-f68259188b89 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:15.582940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 628/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:15.588984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc3a09-9b22-428f-a934-84057638017b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:15.725873 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc3a09-9b22-428f-a934-84057638017b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:15.768654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5a191-8fc4-4759-8719-83d5a8296b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:15.769475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5a191-8fc4-4759-8719-83d5a8296b21 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:15.769760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5a191-8fc4-4759-8719-83d5a8296b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:15.770034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5a191-8fc4-4759-8719-83d5a8296b21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:15.777675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5a191-8fc4-4759-8719-83d5a8296b21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:15.779179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5a191-8fc4-4759-8719-83d5a8296b21 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-2ff5a191-8fc4-4759-8719-83d5a8296b21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:55:15.794830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:15.795994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:15.797143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:55:15.797143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:55:15.797143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:55:15.797143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:55:15.797143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:55:15.797143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:55:15.797143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:15.797143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:15.797143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 630/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:55:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 08 01:55:15.804043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558ac7c-bdf6-4b71-88a0-f68259188b89 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:15.910106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558ac7c-bdf6-4b71-88a0-f68259188b89 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:15.946896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e38507-ce76-4be4-87e4-95576368137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:15.947481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e38507-ce76-4be4-87e4-95576368137e None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:15.947765 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e38507-ce76-4be4-87e4-95576368137e None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:15.948040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e38507-ce76-4be4-87e4-95576368137e None admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:15.948865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e38507-ce76-4be4-87e4-95576368137e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:15.949171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e38507-ce76-4be4-87e4-95576368137e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:15.950666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e38507-ce76-4be4-87e4-95576368137e None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:15.963965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 629/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:55:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 01:55:15.970618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5a191-8fc4-4759-8719-83d5a8296b21 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:15.998639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:16.076364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5a191-8fc4-4759-8719-83d5a8296b21 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:16.110993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efe361-d460-4bc1-b778-53ceb5ed648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:16.111762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efe361-d460-4bc1-b778-53ceb5ed648d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:16.112047 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efe361-d460-4bc1-b778-53ceb5ed648d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:16.112332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efe361-d460-4bc1-b778-53ceb5ed648d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:16.113202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efe361-d460-4bc1-b778-53ceb5ed648d None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=4eb69af145154c6d85bd4b845979e6c0) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:16.113517 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efe361-d460-4bc1-b778-53ceb5ed648d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:16.114980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efe361-d460-4bc1-b778-53ceb5ed648d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:16.120546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 631/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:55:15 2022] GET /identity/v3/role_assignments?user.id=4eb69af145154c6d85bd4b845979e6c0&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:18.321442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ced06-1de2-4c4a-8f38-cb340cd44b67 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:18.321988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ced06-1de2-4c4a-8f38-cb340cd44b67 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:18.321988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ced06-1de2-4c4a-8f38-cb340cd44b67 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:18.323355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 630/1261] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:55:18.328415 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a39142-6299-4571-8084-2caf92a654d4 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:18.328840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a39142-6299-4571-8084-2caf92a654d4 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:18.328981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a39142-6299-4571-8084-2caf92a654d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:18.346652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-12a39142-6299-4571-8084-2caf92a654d4 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:55:18.347844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-12a39142-6299-4571-8084-2caf92a654d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:55:18.413848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 632/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:55:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:55:18.437619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ced06-1de2-4c4a-8f38-cb340cd44b67 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:18.449194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:18.540316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ced06-1de2-4c4a-8f38-cb340cd44b67 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:18.573493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10735eff-f937-4638-b8af-deeff957d77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:18.574217 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10735eff-f937-4638-b8af-deeff957d77d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:18.574502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10735eff-f937-4638-b8af-deeff957d77d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:18.574779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10735eff-f937-4638-b8af-deeff957d77d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:18.585419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10735eff-f937-4638-b8af-deeff957d77d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-10735eff-f937-4638-b8af-deeff957d77d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:55:18.592955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:18.594063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:18.595188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 631/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:55:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:18.600295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a39142-6299-4571-8084-2caf92a654d4 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:18.618190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:18.705288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a39142-6299-4571-8084-2caf92a654d4 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:18.741773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87757ed9-637b-4126-9e08-b984656f4dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:18.742438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757ed9-637b-4126-9e08-b984656f4dcb None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:18.742722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757ed9-637b-4126-9e08-b984656f4dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:18.742998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757ed9-637b-4126-9e08-b984656f4dcb None admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:18.743742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87757ed9-637b-4126-9e08-b984656f4dcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:18.744047 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87757ed9-637b-4126-9e08-b984656f4dcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:18.745454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87757ed9-637b-4126-9e08-b984656f4dcb None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:18.753898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 633/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:55:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:18.759711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10735eff-f937-4638-b8af-deeff957d77d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:18.825850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:18.859627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10735eff-f937-4638-b8af-deeff957d77d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:18.893121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20ffd3-d5bf-451e-88c7-404e4487ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:18.893899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20ffd3-d5bf-451e-88c7-404e4487ec79 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:18.894189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20ffd3-d5bf-451e-88c7-404e4487ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:18.894466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20ffd3-d5bf-451e-88c7-404e4487ec79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:18.904829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20ffd3-d5bf-451e-88c7-404e4487ec79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:18.906324 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20ffd3-d5bf-451e-88c7-404e4487ec79 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-4f20ffd3-d5bf-451e-88c7-404e4487ec79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:18.924688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:18.925589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:18.926652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:18.926652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:18.926652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:18.926652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:18.926652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 632/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:18.931503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87757ed9-637b-4126-9e08-b984656f4dcb None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:19.010855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:19.029950 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87757ed9-637b-4126-9e08-b984656f4dcb None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:19.062474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83bc33d-dae2-4612-b75a-df30c87b8edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:19.063122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bc33d-dae2-4612-b75a-df30c87b8edc None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:19.063408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bc33d-dae2-4612-b75a-df30c87b8edc None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:19.063695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bc33d-dae2-4612-b75a-df30c87b8edc None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:19.064439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83bc33d-dae2-4612-b75a-df30c87b8edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:19.064747 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83bc33d-dae2-4612-b75a-df30c87b8edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:19.066109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83bc33d-dae2-4612-b75a-df30c87b8edc None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:19.078029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 634/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:19.083729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20ffd3-d5bf-451e-88c7-404e4487ec79 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:19.181075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20ffd3-d5bf-451e-88c7-404e4487ec79 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:19.200135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:19.214062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5100c-738b-499a-87df-375989efc5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:19.214846 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5100c-738b-499a-87df-375989efc5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:19.215127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5100c-738b-499a-87df-375989efc5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:19.215401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5100c-738b-499a-87df-375989efc5c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:19.222720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5100c-738b-499a-87df-375989efc5c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:19.224152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5100c-738b-499a-87df-375989efc5c3 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-32f5100c-738b-499a-87df-375989efc5c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:55:19.238492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:19.239578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:19.240756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 633/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:19.245312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83bc33d-dae2-4612-b75a-df30c87b8edc None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:19.343925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83bc33d-dae2-4612-b75a-df30c87b8edc None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:19.378246 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9908a9-b518-4d6f-b945-dd0fc61d3e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:19.378914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9908a9-b518-4d6f-b945-dd0fc61d3e7d None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:19.379203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9908a9-b518-4d6f-b945-dd0fc61d3e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:19.379481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9908a9-b518-4d6f-b945-dd0fc61d3e7d None admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:19.380312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9908a9-b518-4d6f-b945-dd0fc61d3e7d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:19.380651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9908a9-b518-4d6f-b945-dd0fc61d3e7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:19.382088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9908a9-b518-4d6f-b945-dd0fc61d3e7d None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:19.390508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:19.399677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 635/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jun 8 01:55:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:19.405267 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5100c-738b-499a-87df-375989efc5c3 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:19.506981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5100c-738b-499a-87df-375989efc5c3 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:19.540152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edf601-6857-4c4f-a62b-a218cff32653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:19.540954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edf601-6857-4c4f-a62b-a218cff32653 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:19.541235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edf601-6857-4c4f-a62b-a218cff32653 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:19.541508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edf601-6857-4c4f-a62b-a218cff32653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:19.552199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edf601-6857-4c4f-a62b-a218cff32653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:19.553658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97edf601-6857-4c4f-a62b-a218cff32653 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-97edf601-6857-4c4f-a62b-a218cff32653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:19.571002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:19.572025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:19.573188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:19.573188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:19.573188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:19.573188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:19.573188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 634/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:19.577786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9908a9-b518-4d6f-b945-dd0fc61d3e7d None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:19.681623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9908a9-b518-4d6f-b945-dd0fc61d3e7d None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:19.715635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70ca50-c653-4468-913e-1bfa525a543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:19.716289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70ca50-c653-4468-913e-1bfa525a543e None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:19.716604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70ca50-c653-4468-913e-1bfa525a543e None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:19.716883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70ca50-c653-4468-913e-1bfa525a543e None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:19.717595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70ca50-c653-4468-913e-1bfa525a543e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:19.717902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70ca50-c653-4468-913e-1bfa525a543e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:19.719276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70ca50-c653-4468-913e-1bfa525a543e None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:19.730720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 636/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:19.737180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edf601-6857-4c4f-a62b-a218cff32653 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:19.769013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:19.836185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edf601-6857-4c4f-a62b-a218cff32653 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:19.868828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712fc8f-9025-4511-8752-432655c1223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:19.869631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712fc8f-9025-4511-8752-432655c1223d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:19.869913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712fc8f-9025-4511-8752-432655c1223d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:19.870185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712fc8f-9025-4511-8752-432655c1223d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:19.877728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712fc8f-9025-4511-8752-432655c1223d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:19.879180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712fc8f-9025-4511-8752-432655c1223d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-5712fc8f-9025-4511-8752-432655c1223d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:55:19.892988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:19.894055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:19.895235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:55:19.895235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:55:19.895235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:55:19.895235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:55:19.895235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:55:19.895235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:55:19.895235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:19.895235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:19.895235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 635/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:55:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:19.900004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70ca50-c653-4468-913e-1bfa525a543e None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:19.960021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:20.001158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70ca50-c653-4468-913e-1bfa525a543e None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:20.035431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a136496-8949-4fac-a0b1-4cc51e998c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:20.036346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a136496-8949-4fac-a0b1-4cc51e998c77 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:20.036659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a136496-8949-4fac-a0b1-4cc51e998c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:20.036933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a136496-8949-4fac-a0b1-4cc51e998c77 None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:20.037709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a136496-8949-4fac-a0b1-4cc51e998c77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:20.038015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a136496-8949-4fac-a0b1-4cc51e998c77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:20.039438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a136496-8949-4fac-a0b1-4cc51e998c77 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:20.051675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 637/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:55:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:20.057598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712fc8f-9025-4511-8752-432655c1223d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:20.147081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:20.156285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712fc8f-9025-4511-8752-432655c1223d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:20.191637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a865a7-8add-4375-a464-55a14448032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:20.192534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a865a7-8add-4375-a464-55a14448032f None admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:20.192820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a865a7-8add-4375-a464-55a14448032f None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:20.193090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a865a7-8add-4375-a464-55a14448032f None admin] PATH_INFO: `/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/4eb69af145154c6d85bd4b845979e6c0/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:20.194993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a865a7-8add-4375-a464-55a14448032f None admin] RBAC: Authorizing `identity:create_grant(project_id=2fed22cd0ec848c88e021eff03a083f0, user_id=4eb69af145154c6d85bd4b845979e6c0, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:20.196428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a865a7-8add-4375-a464-55a14448032f None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:20.209658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 636/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Jun 8 01:55:20 2022] PUT /identity/v3/projects/2fed22cd0ec848c88e021eff03a083f0/users/4eb69af145154c6d85bd4b845979e6c0/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 01:55:22.461757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dec8e0-fb22-4554-9895-16044f28cab1 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:22.462323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dec8e0-fb22-4554-9895-16044f28cab1 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:22.462323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dec8e0-fb22-4554-9895-16044f28cab1 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:22.463661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 638/1274] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:55:22.468744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5fdf-28a3-48bb-aaf9-91e0d1b01787 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:22.469122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5fdf-28a3-48bb-aaf9-91e0d1b01787 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:22.469289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f5fdf-28a3-48bb-aaf9-91e0d1b01787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:22.488276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-669f5fdf-28a3-48bb-aaf9-91e0d1b01787 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:55:22.489744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-669f5fdf-28a3-48bb-aaf9-91e0d1b01787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:55:22.555603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 637/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:55:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:55:22.579062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dec8e0-fb22-4554-9895-16044f28cab1 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:22.682942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dec8e0-fb22-4554-9895-16044f28cab1 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:22.703748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:22.727363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9479a-6ea2-4ab6-99db-92705a5c6d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:22.728145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9479a-6ea2-4ab6-99db-92705a5c6d46 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:22.728470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9479a-6ea2-4ab6-99db-92705a5c6d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:22.728816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9479a-6ea2-4ab6-99db-92705a5c6d46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:22.739646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9479a-6ea2-4ab6-99db-92705a5c6d46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c3c9479a-6ea2-4ab6-99db-92705a5c6d46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 08 01:55:22.747247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:22.748352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:22.749445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 639/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Jun 8 01:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:22.754748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f5fdf-28a3-48bb-aaf9-91e0d1b01787 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:22.834199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:22.853672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f5fdf-28a3-48bb-aaf9-91e0d1b01787 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:22.886096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2a598-e6d2-4ecb-a65f-1509bf86e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:22.886748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2a598-e6d2-4ecb-a65f-1509bf86e748 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:22.887025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2a598-e6d2-4ecb-a65f-1509bf86e748 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:22.887295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2a598-e6d2-4ecb-a65f-1509bf86e748 None admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:22.888027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2a598-e6d2-4ecb-a65f-1509bf86e748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:22.888327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2a598-e6d2-4ecb-a65f-1509bf86e748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:22.889732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2a598-e6d2-4ecb-a65f-1509bf86e748 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:22.898382 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 638/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Jun 8 01:55:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:22.903973 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9479a-6ea2-4ab6-99db-92705a5c6d46 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:23.001662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9479a-6ea2-4ab6-99db-92705a5c6d46 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:23.042120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414d732-c9f1-4e97-ba46-90859427647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:23.042902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414d732-c9f1-4e97-ba46-90859427647b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:23.043186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414d732-c9f1-4e97-ba46-90859427647b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:23.043458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414d732-c9f1-4e97-ba46-90859427647b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:23.054057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414d732-c9f1-4e97-ba46-90859427647b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:23.055520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414d732-c9f1-4e97-ba46-90859427647b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-c414d732-c9f1-4e97-ba46-90859427647b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:23.074043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:23.075227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:23.076363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:23.076363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:23.076363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:23.076363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:23.076363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 640/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:23.081581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2a598-e6d2-4ecb-a65f-1509bf86e748 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:23.179147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2a598-e6d2-4ecb-a65f-1509bf86e748 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:23.211542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80268d4e-72dc-47f7-b995-0042741a4267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:23.212193 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80268d4e-72dc-47f7-b995-0042741a4267 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:23.212505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80268d4e-72dc-47f7-b995-0042741a4267 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:23.212782 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80268d4e-72dc-47f7-b995-0042741a4267 None admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:23.213494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80268d4e-72dc-47f7-b995-0042741a4267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:23.213799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80268d4e-72dc-47f7-b995-0042741a4267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:23.215163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80268d4e-72dc-47f7-b995-0042741a4267 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:23.227081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 639/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:23.232461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414d732-c9f1-4e97-ba46-90859427647b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:23.263564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:23.332305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414d732-c9f1-4e97-ba46-90859427647b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:23.365405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1ea0e-52c2-4fd4-af1d-3520bb688496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:23.366192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1ea0e-52c2-4fd4-af1d-3520bb688496 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:23.366479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1ea0e-52c2-4fd4-af1d-3520bb688496 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:23.366759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1ea0e-52c2-4fd4-af1d-3520bb688496 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:23.373987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1ea0e-52c2-4fd4-af1d-3520bb688496 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:23.375433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1ea0e-52c2-4fd4-af1d-3520bb688496 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-32b1ea0e-52c2-4fd4-af1d-3520bb688496 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 08 01:55:23.389409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:23.390437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:23.391672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 641/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:23.396239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80268d4e-72dc-47f7-b995-0042741a4267 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:23.399989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:23.494661 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80268d4e-72dc-47f7-b995-0042741a4267 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:23.527302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99f7bc-d2ee-4da4-85c0-087802212908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:23.527902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99f7bc-d2ee-4da4-85c0-087802212908 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:23.528180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99f7bc-d2ee-4da4-85c0-087802212908 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:23.528544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99f7bc-d2ee-4da4-85c0-087802212908 None admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:23.529363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99f7bc-d2ee-4da4-85c0-087802212908 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:23.529668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99f7bc-d2ee-4da4-85c0-087802212908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:23.531094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99f7bc-d2ee-4da4-85c0-087802212908 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:23.548232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 640/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Jun 8 01:55:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:23.553915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1ea0e-52c2-4fd4-af1d-3520bb688496 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:23.640830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:23.650621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1ea0e-52c2-4fd4-af1d-3520bb688496 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:23.687267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aba23-61e4-414f-8c8c-eeef40810e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:23.687902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aba23-61e4-414f-8c8c-eeef40810e05 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:23.688181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aba23-61e4-414f-8c8c-eeef40810e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:23.688482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aba23-61e4-414f-8c8c-eeef40810e05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:23.753491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aba23-61e4-414f-8c8c-eeef40810e05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:23.754926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aba23-61e4-414f-8c8c-eeef40810e05 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:23.839360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:23.849662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 642/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Jun 8 01:55:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 08 01:55:23.855376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99f7bc-d2ee-4da4-85c0-087802212908 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:23.916787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:23.956088 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99f7bc-d2ee-4da4-85c0-087802212908 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:23.989716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827e987-2ebb-49e4-9e4b-e8f202187eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:23.990539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827e987-2ebb-49e4-9e4b-e8f202187eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:23.990824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827e987-2ebb-49e4-9e4b-e8f202187eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:23.991100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827e987-2ebb-49e4-9e4b-e8f202187eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:24.000989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827e987-2ebb-49e4-9e4b-e8f202187eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:24.002439 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827e987-2ebb-49e4-9e4b-e8f202187eb9 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-d827e987-2ebb-49e4-9e4b-e8f202187eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 08 01:55:24.019483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:24.020580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:24.021855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 08 01:55:24.021855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 08 01:55:24.021855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 08 01:55:24.021855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:24.021855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 641/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:24.026335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aba23-61e4-414f-8c8c-eeef40810e05 None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:24.123850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aba23-61e4-414f-8c8c-eeef40810e05 None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:24.156235 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a183e3d-a586-4846-9471-57788a0a576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:24.156913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a183e3d-a586-4846-9471-57788a0a576a None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:24.157198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a183e3d-a586-4846-9471-57788a0a576a None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:24.157473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a183e3d-a586-4846-9471-57788a0a576a None admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:24.158793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a183e3d-a586-4846-9471-57788a0a576a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:24.159203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a183e3d-a586-4846-9471-57788a0a576a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:24.160595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a183e3d-a586-4846-9471-57788a0a576a None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:24.171699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 643/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Jun 8 01:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:24.177704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827e987-2ebb-49e4-9e4b-e8f202187eb9 None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:24.275701 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827e987-2ebb-49e4-9e4b-e8f202187eb9 None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:24.289553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:24.309649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ea29d-eda0-42c2-bd47-52c09165c8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:24.310426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ea29d-eda0-42c2-bd47-52c09165c8fd None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:24.310708 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ea29d-eda0-42c2-bd47-52c09165c8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:24.310982 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ea29d-eda0-42c2-bd47-52c09165c8fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:24.317582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ea29d-eda0-42c2-bd47-52c09165c8fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:24.319033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ea29d-eda0-42c2-bd47-52c09165c8fd None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application [None req-a30ea29d-eda0-42c2-bd47-52c09165c8fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 08 01:55:24.332974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:24.334062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:24.335179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 08 01:55:24.335179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 08 01:55:24.335179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 08 01:55:24.335179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 08 01:55:24.335179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 08 01:55:24.335179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 08 01:55:24.335179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 08 01:55:24.335179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: ERROR keystone.server.flask.application Jun 08 01:55:24.335179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 642/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Jun 8 01:55:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:24.339827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a183e3d-a586-4846-9471-57788a0a576a None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:24.389518 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:24.438539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a183e3d-a586-4846-9471-57788a0a576a None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:24.471513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7c6fd-b2af-4f8b-a676-90f692e4b1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:24.472160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7c6fd-b2af-4f8b-a676-90f692e4b1ec None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:24.472472 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7c6fd-b2af-4f8b-a676-90f692e4b1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:24.472753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7c6fd-b2af-4f8b-a676-90f692e4b1ec None admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:24.473533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7c6fd-b2af-4f8b-a676-90f692e4b1ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:24.473846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7c6fd-b2af-4f8b-a676-90f692e4b1ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:24.475274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7c6fd-b2af-4f8b-a676-90f692e4b1ec None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:24.487601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 644/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Jun 8 01:55:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:24.493755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ea29d-eda0-42c2-bd47-52c09165c8fd None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:24.590895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ea29d-eda0-42c2-bd47-52c09165c8fd None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:24.623336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27d6d9-723b-4c6e-8ddd-d5ed2850a19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:24.623980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d6d9-723b-4c6e-8ddd-d5ed2850a19b None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:24.624261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d6d9-723b-4c6e-8ddd-d5ed2850a19b None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:24.624566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27d6d9-723b-4c6e-8ddd-d5ed2850a19b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:24.625385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27d6d9-723b-4c6e-8ddd-d5ed2850a19b None admin] RBAC: Adding query filter params (role.id=71044d1835ec4f909f32dbffcfe31811, scope.project.id=2fed22cd0ec848c88e021eff03a083f0, user.id=4eb69af145154c6d85bd4b845979e6c0) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:24.625693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27d6d9-723b-4c6e-8ddd-d5ed2850a19b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:24.627639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27d6d9-723b-4c6e-8ddd-d5ed2850a19b None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:24.633327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 643/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Jun 8 01:55:24 2022] GET /identity/v3/role_assignments?user.id=4eb69af145154c6d85bd4b845979e6c0&scope.project.id=2fed22cd0ec848c88e021eff03a083f0&role.id=71044d1835ec4f909f32dbffcfe31811 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:26.882526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437814-b984-4eb4-bf13-00cb655d78a7 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:26.883080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437814-b984-4eb4-bf13-00cb655d78a7 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:26.883080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437814-b984-4eb4-bf13-00cb655d78a7 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:26.884453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 645/1288] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:55:26.889665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784637d9-f3ac-46fc-97e7-d5265fd95004 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:26.890097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784637d9-f3ac-46fc-97e7-d5265fd95004 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:26.890307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784637d9-f3ac-46fc-97e7-d5265fd95004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:26.906687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-784637d9-f3ac-46fc-97e7-d5265fd95004 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:55:26.907892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-784637d9-f3ac-46fc-97e7-d5265fd95004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:55:26.937877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:26.974202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 644/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:55:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:55:26.996891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437814-b984-4eb4-bf13-00cb655d78a7 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:27.096471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437814-b984-4eb4-bf13-00cb655d78a7 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:27.106779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:27.130269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c905831-e035-4094-ae58-1d11f2bcefaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:27.131050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c905831-e035-4094-ae58-1d11f2bcefaa None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:27.131326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c905831-e035-4094-ae58-1d11f2bcefaa None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:27.131600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c905831-e035-4094-ae58-1d11f2bcefaa None admin] PATH_INFO: `/v3/services/placement` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:27.132241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c905831-e035-4094-ae58-1d11f2bcefaa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:27.133688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c905831-e035-4094-ae58-1d11f2bcefaa None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-4c905831-e035-4094-ae58-1d11f2bcefaa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:27.144972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:27.146190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:27.147298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 646/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jun 8 01:55:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:27.152290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784637d9-f3ac-46fc-97e7-d5265fd95004 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:27.249812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784637d9-f3ac-46fc-97e7-d5265fd95004 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:27.283003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f68ee-54df-47d5-8680-ae0524f523f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:27.283912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f68ee-54df-47d5-8680-ae0524f523f0 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:27.284196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f68ee-54df-47d5-8680-ae0524f523f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:27.284501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f68ee-54df-47d5-8680-ae0524f523f0 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:27.285197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f68ee-54df-47d5-8680-ae0524f523f0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:27.285501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f68ee-54df-47d5-8680-ae0524f523f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:27.287132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f68ee-54df-47d5-8680-ae0524f523f0 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:27.291876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 645/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Jun 8 01:55:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:27.297735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c905831-e035-4094-ae58-1d11f2bcefaa None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:27.397525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c905831-e035-4094-ae58-1d11f2bcefaa None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:27.430084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385fa20-7724-4e81-a428-1bcc628d4155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:27.430737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385fa20-7724-4e81-a428-1bcc628d4155 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:27.431022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385fa20-7724-4e81-a428-1bcc628d4155 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:27.431298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385fa20-7724-4e81-a428-1bcc628d4155 None admin] PATH_INFO: `/v3/services` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:27.432002 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385fa20-7724-4e81-a428-1bcc628d4155 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:27.432308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385fa20-7724-4e81-a428-1bcc628d4155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:27.435515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385fa20-7724-4e81-a428-1bcc628d4155 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:27.440576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 647/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Jun 8 01:55:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:29.690550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95221b29-8f5d-43f5-882f-826b7bca2d8a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:29.691219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95221b29-8f5d-43f5-882f-826b7bca2d8a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:29.691356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95221b29-8f5d-43f5-882f-826b7bca2d8a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:29.693009 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 646/1293] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:55:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:55:29.699091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6f6a8-8d33-415d-ae6a-f92ca6f78ddb None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:29.699546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6f6a8-8d33-415d-ae6a-f92ca6f78ddb None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:29.699824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6f6a8-8d33-415d-ae6a-f92ca6f78ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:29.718206 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-45a6f6a8-8d33-415d-ae6a-f92ca6f78ddb None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:55:29.719739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-45a6f6a8-8d33-415d-ae6a-f92ca6f78ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:55:29.753352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:29.790584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 648/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:55:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:55:29.813505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95221b29-8f5d-43f5-882f-826b7bca2d8a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:29.844590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:29.914449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95221b29-8f5d-43f5-882f-826b7bca2d8a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:29.948443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5256e52-3fe0-4da4-b234-1a51a9ab9087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:29.949050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5256e52-3fe0-4da4-b234-1a51a9ab9087 None admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:29.949330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5256e52-3fe0-4da4-b234-1a51a9ab9087 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:29.949603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5256e52-3fe0-4da4-b234-1a51a9ab9087 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:29.950313 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5256e52-3fe0-4da4-b234-1a51a9ab9087 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:29.951662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5256e52-3fe0-4da4-b234-1a51a9ab9087 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:29.969392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 647/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Jun 8 01:55:29 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:55:32.183155 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4326ddd-439d-4aaf-82a7-ef083dc956f5 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:32.183672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4326ddd-439d-4aaf-82a7-ef083dc956f5 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:32.183672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4326ddd-439d-4aaf-82a7-ef083dc956f5 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:32.184998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 649/1296] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:55:32.190740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8798a-c02b-469d-915e-6872c4fb21c9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:32.191130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8798a-c02b-469d-915e-6872c4fb21c9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:32.191290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8798a-c02b-469d-915e-6872c4fb21c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:32.208158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d4d8798a-c02b-469d-915e-6872c4fb21c9 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:55:32.209373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d4d8798a-c02b-469d-915e-6872c4fb21c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:55:32.279584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 648/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:55:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 08 01:55:32.302201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4326ddd-439d-4aaf-82a7-ef083dc956f5 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:32.407265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4326ddd-439d-4aaf-82a7-ef083dc956f5 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:32.442464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ef236-b644-4f5b-9d99-ed74ecff515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:32.443184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ef236-b644-4f5b-9d99-ed74ecff515b None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:32.443462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ef236-b644-4f5b-9d99-ed74ecff515b None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:32.443731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ef236-b644-4f5b-9d99-ed74ecff515b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:32.444366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ef236-b644-4f5b-9d99-ed74ecff515b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:32.445757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ef236-b644-4f5b-9d99-ed74ecff515b None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-2e6ef236-b644-4f5b-9d99-ed74ecff515b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:32.457793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:32.458850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:32.459776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 650/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jun 8 01:55:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:32.465116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8798a-c02b-469d-915e-6872c4fb21c9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:32.483061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:32.563817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8798a-c02b-469d-915e-6872c4fb21c9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:32.596943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f6f77-11cb-48b5-be57-b64b5165c76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:32.597598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f6f77-11cb-48b5-be57-b64b5165c76d None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:32.597882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f6f77-11cb-48b5-be57-b64b5165c76d None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:32.598161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f6f77-11cb-48b5-be57-b64b5165c76d None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:32.598860 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f6f77-11cb-48b5-be57-b64b5165c76d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:32.599167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f6f77-11cb-48b5-be57-b64b5165c76d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:32.600581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f6f77-11cb-48b5-be57-b64b5165c76d None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:32.605614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 649/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Jun 8 01:55:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:32.611882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ef236-b644-4f5b-9d99-ed74ecff515b None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:32.643373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:32.713239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ef236-b644-4f5b-9d99-ed74ecff515b None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:32.745723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62806095-1c98-4b09-9786-bc8bb839bab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:32.746370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62806095-1c98-4b09-9786-bc8bb839bab5 None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:32.746656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62806095-1c98-4b09-9786-bc8bb839bab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:32.746934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62806095-1c98-4b09-9786-bc8bb839bab5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:32.747720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62806095-1c98-4b09-9786-bc8bb839bab5 None admin] RBAC: Adding query filter params (interface=public, service_id=5d9b16dff6414bd28201fbd90d49e03f, region_id=RegionOne) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:32.748036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62806095-1c98-4b09-9786-bc8bb839bab5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:32.749450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62806095-1c98-4b09-9786-bc8bb839bab5 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:32.755030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 651/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Jun 8 01:55:32 2022] GET /identity/v3/endpoints?service_id=5d9b16dff6414bd28201fbd90d49e03f&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:32.760630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f6f77-11cb-48b5-be57-b64b5165c76d None admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:32.839996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:32.858359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f6f77-11cb-48b5-be57-b64b5165c76d None admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:32.890877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ea349-ae64-4d3b-84ea-bce3f904aa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:32.891528 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ea349-ae64-4d3b-84ea-bce3f904aa00 None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:32.891808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ea349-ae64-4d3b-84ea-bce3f904aa00 None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:32.892080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ea349-ae64-4d3b-84ea-bce3f904aa00 None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:32.892836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ea349-ae64-4d3b-84ea-bce3f904aa00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:32.894353 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ea349-ae64-4d3b-84ea-bce3f904aa00 None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:32.900966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 650/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Jun 8 01:55:32 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 01:55:35.122633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd20e-b4ce-4613-be8d-f82fbd97d75f None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:35.123193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd20e-b4ce-4613-be8d-f82fbd97d75f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:35.123193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd20e-b4ce-4613-be8d-f82fbd97d75f None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:35.124571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 652/1302] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:55:35.129766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3e3e-395b-49f3-996e-15f11aa87beb None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:35.130188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3e3e-395b-49f3-996e-15f11aa87beb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:35.130323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3e3e-395b-49f3-996e-15f11aa87beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:35.146763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-404b3e3e-395b-49f3-996e-15f11aa87beb None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:55:35.147936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-404b3e3e-395b-49f3-996e-15f11aa87beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:55:35.213449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 651/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:55:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 08 01:55:35.237231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcd20e-b4ce-4613-be8d-f82fbd97d75f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:35.333329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:35.340239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcd20e-b4ce-4613-be8d-f82fbd97d75f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:35.373528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fbf51-7383-4b75-be96-66308e20398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:35.374318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fbf51-7383-4b75-be96-66308e20398c None admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:35.374600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fbf51-7383-4b75-be96-66308e20398c None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:35.374874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fbf51-7383-4b75-be96-66308e20398c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:35.375515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fbf51-7383-4b75-be96-66308e20398c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:35.376947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fbf51-7383-4b75-be96-66308e20398c None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application [None req-ad8fbf51-7383-4b75-be96-66308e20398c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 08 01:55:35.388665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 08 01:55:35.389856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.get_or_create( Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application with Lock( Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self._enter() Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application return self.creator() Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application created_value = creator( Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: ERROR keystone.server.flask.application Jun 08 01:55:35.390711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 653/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Jun 8 01:55:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 08 01:55:35.396249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3e3e-395b-49f3-996e-15f11aa87beb None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:35.471931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:35.494362 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3e3e-395b-49f3-996e-15f11aa87beb None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:35.528991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872f21d-42b5-483a-a997-dbf0e6b5d5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:35.529649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f21d-42b5-483a-a997-dbf0e6b5d5cc None admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:35.529930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f21d-42b5-483a-a997-dbf0e6b5d5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:35.530200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f21d-42b5-483a-a997-dbf0e6b5d5cc None admin] PATH_INFO: `/v3/services` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:35.530886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872f21d-42b5-483a-a997-dbf0e6b5d5cc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 01:55:35.531185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872f21d-42b5-483a-a997-dbf0e6b5d5cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:35.532567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872f21d-42b5-483a-a997-dbf0e6b5d5cc None admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:35.538506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 652/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Jun 8 01:55:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 01:55:35.544330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fbf51-7383-4b75-be96-66308e20398c None admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:55:35.641375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fbf51-7383-4b75-be96-66308e20398c None admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:55:35.675601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14570c-146d-4287-ad01-8f085656d1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:55:35.676235 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14570c-146d-4287-ad01-8f085656d1b6 None admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:55:35.676547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14570c-146d-4287-ad01-8f085656d1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:55:35.676826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14570c-146d-4287-ad01-8f085656d1b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:55:35.677547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14570c-146d-4287-ad01-8f085656d1b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:55:35.678904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14570c-146d-4287-ad01-8f085656d1b6 None admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:55:35.698197 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 654/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Jun 8 01:55:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 01:55:45.535247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=30 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:55:45.694177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:39.813107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fa360-62b5-46be-a5ef-e4db9a0eb435 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:39.813540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fa360-62b5-46be-a5ef-e4db9a0eb435 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:39.813540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fa360-62b5-46be-a5ef-e4db9a0eb435 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:39.820899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 653/1307] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:56:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:56:39.822768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa13f8-fa11-4d6a-8332-3891722c5aa0 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:39.823219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa13f8-fa11-4d6a-8332-3891722c5aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:39.823301 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa13f8-fa11-4d6a-8332-3891722c5aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:39.912859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-48fa13f8-fa11-4d6a-8332-3891722c5aa0 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:56:39.914091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-48fa13f8-fa11-4d6a-8332-3891722c5aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:40.027177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 655/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:56:40.102751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af51cb-e55e-4931-a3a2-c98ed82a4005 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:40.103049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af51cb-e55e-4931-a3a2-c98ed82a4005 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:40.103335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af51cb-e55e-4931-a3a2-c98ed82a4005 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:40.104732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 654/1309] 10.222.0.20 () {54 vars in 964 bytes} [Wed Jun 8 01:56:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:56:40.112970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d1c5-9cb9-400e-aeef-92311c699ae0 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:40.113266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d1c5-9cb9-400e-aeef-92311c699ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:40.113530 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83d1c5-9cb9-400e-aeef-92311c699ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:40.235689 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-2b83d1c5-9cb9-400e-aeef-92311c699ae0 None None] MFA Rules not processed for user `d26f4426ba8c43399c4a7ef2f1701f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:56:40.237326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2b83d1c5-9cb9-400e-aeef-92311c699ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:40.259872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:40.337509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 656/1310] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Jun 8 01:56:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:56:40.347477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af51cb-e55e-4931-a3a2-c98ed82a4005 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:56:40.349823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-79af51cb-e55e-4931-a3a2-c98ed82a4005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:40.472807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af51cb-e55e-4931-a3a2-c98ed82a4005 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:56:40.557598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39796f-f170-453a-9f39-715807f186cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:56:40.558628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39796f-f170-453a-9f39-715807f186cb service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:40.559329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39796f-f170-453a-9f39-715807f186cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:40.559859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39796f-f170-453a-9f39-715807f186cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:40.561728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0e39796f-f170-453a-9f39-715807f186cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:40.628089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:40.693236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39796f-f170-453a-9f39-715807f186cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:56:40.695046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39796f-f170-453a-9f39-715807f186cb service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:56:40.716758 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 655/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:56:43.355234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20418d-069b-4fbe-9d6a-4df687b74f4e None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:43.355780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20418d-069b-4fbe-9d6a-4df687b74f4e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:43.355780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20418d-069b-4fbe-9d6a-4df687b74f4e None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:43.360474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 657/1312] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:56:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:56:43.366115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1243aaa-08f5-4136-a04e-465b11d97ff1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:43.366565 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1243aaa-08f5-4136-a04e-465b11d97ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:43.367133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1243aaa-08f5-4136-a04e-465b11d97ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:43.452066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e1243aaa-08f5-4136-a04e-465b11d97ff1 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:56:43.453314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e1243aaa-08f5-4136-a04e-465b11d97ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:43.557318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 656/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:56:43.613239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552aac0-77e4-45d5-826c-a2be242bba1f None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:43.613528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552aac0-77e4-45d5-826c-a2be242bba1f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:43.613815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552aac0-77e4-45d5-826c-a2be242bba1f None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:43.615329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 658/1314] 10.222.0.20 () {54 vars in 964 bytes} [Wed Jun 8 01:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:56:43.621416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7894b-072c-41bc-8e13-62e09db677ab None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:43.621708 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7894b-072c-41bc-8e13-62e09db677ab None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:43.621987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7894b-072c-41bc-8e13-62e09db677ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:43.635192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:43.708238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-67a7894b-072c-41bc-8e13-62e09db677ab None None] MFA Rules not processed for user `d26f4426ba8c43399c4a7ef2f1701f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:56:43.709431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-67a7894b-072c-41bc-8e13-62e09db677ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:43.785313 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 657/1315] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Jun 8 01:56:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:56:43.794460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552aac0-77e4-45d5-826c-a2be242bba1f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:56:43.795974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b552aac0-77e4-45d5-826c-a2be242bba1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:43.897890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552aac0-77e4-45d5-826c-a2be242bba1f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:56:43.971958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:43.974558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c797b-432c-4a5d-8a48-64c4bb606b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:56:43.975384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c797b-432c-4a5d-8a48-64c4bb606b88 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:43.975953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c797b-432c-4a5d-8a48-64c4bb606b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:43.976380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c797b-432c-4a5d-8a48-64c4bb606b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:43.978131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c88c797b-432c-4a5d-8a48-64c4bb606b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:44.093939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c797b-432c-4a5d-8a48-64c4bb606b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:56:44.096057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c797b-432c-4a5d-8a48-64c4bb606b88 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:56:44.115421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 659/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:56:46.919264 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d047f55-0f2d-4711-9635-48ff1615c30d None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:46.919726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d047f55-0f2d-4711-9635-48ff1615c30d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:46.919824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d047f55-0f2d-4711-9635-48ff1615c30d None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:46.921282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 658/1317] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:56:46.927114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a87ab-81e7-4734-80b9-c18481635df8 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:46.927527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a87ab-81e7-4734-80b9-c18481635df8 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:46.927667 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a87ab-81e7-4734-80b9-c18481635df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:46.990899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-1b3a87ab-81e7-4734-80b9-c18481635df8 None None] MFA Rules not processed for user `9dd12833b3ae4101a15287c432ec51b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:56:46.992113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1b3a87ab-81e7-4734-80b9-c18481635df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:47.044056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:47.090595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 660/1318] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:56:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:56:47.130376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d047f55-0f2d-4711-9635-48ff1615c30d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:56:47.131971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9d047f55-0f2d-4711-9635-48ff1615c30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:47.236253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d047f55-0f2d-4711-9635-48ff1615c30d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:56:47.239118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:47.309080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e54a9-86ff-42f7-aab2-7e143d0335fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:56:47.309731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e54a9-86ff-42f7-aab2-7e143d0335fd service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:47.310011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e54a9-86ff-42f7-aab2-7e143d0335fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:47.310285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e54a9-86ff-42f7-aab2-7e143d0335fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:47.311913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ff0e54a9-86ff-42f7-aab2-7e143d0335fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:47.418747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e54a9-86ff-42f7-aab2-7e143d0335fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:56:47.420187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e54a9-86ff-42f7-aab2-7e143d0335fd service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:56:47.438417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 659/1319] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:56:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 01:56:50.231135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70906998-63d0-497c-87d4-fca6d4fc56f9 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:50.231658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70906998-63d0-497c-87d4-fca6d4fc56f9 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:50.231658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70906998-63d0-497c-87d4-fca6d4fc56f9 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:50.233012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 661/1320] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:56:50.238788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78d8a8-73b0-4c11-9caa-454e4a8cee24 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:50.239228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78d8a8-73b0-4c11-9caa-454e4a8cee24 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:50.239346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78d8a8-73b0-4c11-9caa-454e4a8cee24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:50.278626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:50.304542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5d78d8a8-73b0-4c11-9caa-454e4a8cee24 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:56:50.305759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5d78d8a8-73b0-4c11-9caa-454e4a8cee24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:50.401842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 660/1321] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:56:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:56:50.441528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70906998-63d0-497c-87d4-fca6d4fc56f9 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:56:50.443154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-70906998-63d0-497c-87d4-fca6d4fc56f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:50.546023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70906998-63d0-497c-87d4-fca6d4fc56f9 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:56:50.619591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5062185-3ecd-4a67-8388-a4687f221877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:56:50.620236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5062185-3ecd-4a67-8388-a4687f221877 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:50.620554 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5062185-3ecd-4a67-8388-a4687f221877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:50.620837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5062185-3ecd-4a67-8388-a4687f221877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:50.622483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a5062185-3ecd-4a67-8388-a4687f221877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:50.630410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:50.733289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5062185-3ecd-4a67-8388-a4687f221877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:56:50.734730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5062185-3ecd-4a67-8388-a4687f221877 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:56:50.753277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 662/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:56:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:56:53.915762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa2d2-b67f-4769-97bc-41572d6932d7 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:53.916191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa2d2-b67f-4769-97bc-41572d6932d7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:53.916437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa2d2-b67f-4769-97bc-41572d6932d7 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:53.917720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 661/1323] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:56:53.923277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf72b78-c884-4856-8813-d9b2c1af1437 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:53.923734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf72b78-c884-4856-8813-d9b2c1af1437 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:53.923989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf72b78-c884-4856-8813-d9b2c1af1437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:53.988295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-adf72b78-c884-4856-8813-d9b2c1af1437 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:56:53.989507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-adf72b78-c884-4856-8813-d9b2c1af1437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:54.047594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:54.087148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 663/1324] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:56:54.124409 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aa2d2-b67f-4769-97bc-41572d6932d7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:56:54.126116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-446aa2d2-b67f-4769-97bc-41572d6932d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:54.230535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aa2d2-b67f-4769-97bc-41572d6932d7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:56:54.242882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:54.309814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ddae1-bc63-4336-9125-e0f2d419f51e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:56:54.310466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ddae1-bc63-4336-9125-e0f2d419f51e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:54.310749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ddae1-bc63-4336-9125-e0f2d419f51e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:54.311025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ddae1-bc63-4336-9125-e0f2d419f51e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:54.312703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-225ddae1-bc63-4336-9125-e0f2d419f51e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:54.555048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ddae1-bc63-4336-9125-e0f2d419f51e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:56:54.556530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ddae1-bc63-4336-9125-e0f2d419f51e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:56:54.575156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 662/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:56:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:56:56.895237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538d743-63e2-4633-8c25-a547b4d8a4e8 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:56.895776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538d743-63e2-4633-8c25-a547b4d8a4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:56.895776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538d743-63e2-4633-8c25-a547b4d8a4e8 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:56.897196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 664/1326] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:56:56.902550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bf75d-fe60-49fb-8c25-ea18cd7740b2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:56.902978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bf75d-fe60-49fb-8c25-ea18cd7740b2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:56.903113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bf75d-fe60-49fb-8c25-ea18cd7740b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:56.953900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:56.972969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fe4bf75d-fe60-49fb-8c25-ea18cd7740b2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:56:56.974184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fe4bf75d-fe60-49fb-8c25-ea18cd7740b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:57.070823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 663/1327] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:56:57.105419 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538d743-63e2-4633-8c25-a547b4d8a4e8 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:56:57.106938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4538d743-63e2-4633-8c25-a547b4d8a4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:57.211980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538d743-63e2-4633-8c25-a547b4d8a4e8 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:56:57.284607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd41629-ba33-406f-8955-2827fa19acdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:56:57.285253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd41629-ba33-406f-8955-2827fa19acdf service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:57.285533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd41629-ba33-406f-8955-2827fa19acdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:57.285809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd41629-ba33-406f-8955-2827fa19acdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:57.287436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4cd41629-ba33-406f-8955-2827fa19acdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:57.305183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:57.403645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd41629-ba33-406f-8955-2827fa19acdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:56:57.405135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd41629-ba33-406f-8955-2827fa19acdf service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:56:57.423907 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 665/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:56:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:56:59.768094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aba1de-c175-4ab6-8f3e-212c4c1b5f6a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:59.768660 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aba1de-c175-4ab6-8f3e-212c4c1b5f6a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:59.768779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aba1de-c175-4ab6-8f3e-212c4c1b5f6a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:59.770099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 664/1329] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:56:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:56:59.775481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95533250-dcf8-4312-9bf4-0cb41f08ec21 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:56:59.775878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95533250-dcf8-4312-9bf4-0cb41f08ec21 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:56:59.776051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95533250-dcf8-4312-9bf4-0cb41f08ec21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:56:59.842110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-95533250-dcf8-4312-9bf4-0cb41f08ec21 None None] MFA Rules not processed for user `9dd12833b3ae4101a15287c432ec51b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:56:59.843326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-95533250-dcf8-4312-9bf4-0cb41f08ec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:56:59.915238 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:56:59.963239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 666/1330] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:56:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:00.003271 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aba1de-c175-4ab6-8f3e-212c4c1b5f6a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:00.004951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-29aba1de-c175-4ab6-8f3e-212c4c1b5f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:00.109275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aba1de-c175-4ab6-8f3e-212c4c1b5f6a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:00.130459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:00.185597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5aa277-50b0-4fc9-ad95-df4ace40523d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:00.186252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aa277-50b0-4fc9-ad95-df4ace40523d service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:00.186538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aa277-50b0-4fc9-ad95-df4ace40523d service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:00.186812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aa277-50b0-4fc9-ad95-df4ace40523d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:00.189010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-bd5aa277-50b0-4fc9-ad95-df4ace40523d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:00.301107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5aa277-50b0-4fc9-ad95-df4ace40523d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:00.302462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5aa277-50b0-4fc9-ad95-df4ace40523d service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:00.320869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 665/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:02.845181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76758a3b-2b57-43c3-b6e6-5da67400f28a None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:02.845704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76758a3b-2b57-43c3-b6e6-5da67400f28a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:02.845817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76758a3b-2b57-43c3-b6e6-5da67400f28a None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:02.847186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 667/1332] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:02.852579 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e6a2-9c76-4a32-8262-053bf8464824 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:02.852999 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e6a2-9c76-4a32-8262-053bf8464824 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:02.853136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e6a2-9c76-4a32-8262-053bf8464824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:02.915755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:02.918561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0d72e6a2-9c76-4a32-8262-053bf8464824 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:02.920289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0d72e6a2-9c76-4a32-8262-053bf8464824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:03.016168 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 666/1333] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:03.054268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76758a3b-2b57-43c3-b6e6-5da67400f28a None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:03.055805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-76758a3b-2b57-43c3-b6e6-5da67400f28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:03.162662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76758a3b-2b57-43c3-b6e6-5da67400f28a None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:03.238514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b65a43-a07d-4cf8-921b-8cde8f0dfa7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:03.239157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b65a43-a07d-4cf8-921b-8cde8f0dfa7d service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:03.239439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b65a43-a07d-4cf8-921b-8cde8f0dfa7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:03.239713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b65a43-a07d-4cf8-921b-8cde8f0dfa7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:03.241366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-99b65a43-a07d-4cf8-921b-8cde8f0dfa7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:03.269111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:03.355458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b65a43-a07d-4cf8-921b-8cde8f0dfa7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:03.356955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b65a43-a07d-4cf8-921b-8cde8f0dfa7d service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:03.376712 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 668/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:05.698575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d66189-ffde-46e5-aad8-8e3700d7f8a5 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:05.699173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d66189-ffde-46e5-aad8-8e3700d7f8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:05.699173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d66189-ffde-46e5-aad8-8e3700d7f8a5 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:05.700533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 667/1335] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:05.705877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea1242-4adb-4d5c-ac5d-440665700bd3 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:05.706303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea1242-4adb-4d5c-ac5d-440665700bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:05.706432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea1242-4adb-4d5c-ac5d-440665700bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:05.773943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-4fea1242-4adb-4d5c-ac5d-440665700bd3 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:05.775159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4fea1242-4adb-4d5c-ac5d-440665700bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:05.850613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:05.872704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 669/1336] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:05.913819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d66189-ffde-46e5-aad8-8e3700d7f8a5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:05.915397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-34d66189-ffde-46e5-aad8-8e3700d7f8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:06.018268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d66189-ffde-46e5-aad8-8e3700d7f8a5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:06.046765 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:06.093286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3a59e-3b03-4248-9536-555c02f33835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:06.093945 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a59e-3b03-4248-9536-555c02f33835 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:06.094227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a59e-3b03-4248-9536-555c02f33835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:06.095048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a59e-3b03-4248-9536-555c02f33835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:06.096738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e1f3a59e-3b03-4248-9536-555c02f33835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:06.209697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3a59e-3b03-4248-9536-555c02f33835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:06.211145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3a59e-3b03-4248-9536-555c02f33835 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:06.230721 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 668/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:08.854452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147120-37d0-4aaf-836c-931a05274173 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:08.854991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147120-37d0-4aaf-836c-931a05274173 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:08.854991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147120-37d0-4aaf-836c-931a05274173 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:08.856360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 670/1338] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:08.862176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4208448-db74-4170-ab55-568120d6d709 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:08.862612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4208448-db74-4170-ab55-568120d6d709 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:08.862714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4208448-db74-4170-ab55-568120d6d709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:08.935135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a4208448-db74-4170-ab55-568120d6d709 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:08.936414 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a4208448-db74-4170-ab55-568120d6d709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:08.954683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:09.043506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 669/1339] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:57:09.079584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8147120-37d0-4aaf-836c-931a05274173 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:09.081477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e8147120-37d0-4aaf-836c-931a05274173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:09.188013 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8147120-37d0-4aaf-836c-931a05274173 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:09.259829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987934a-b5d4-4fd3-92b2-a5454876c50c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:09.260494 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987934a-b5d4-4fd3-92b2-a5454876c50c service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:09.260774 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987934a-b5d4-4fd3-92b2-a5454876c50c service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:09.261053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987934a-b5d4-4fd3-92b2-a5454876c50c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:09.262680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f987934a-b5d4-4fd3-92b2-a5454876c50c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:09.299750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:09.372668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987934a-b5d4-4fd3-92b2-a5454876c50c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:09.374117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f987934a-b5d4-4fd3-92b2-a5454876c50c service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:09.520057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 671/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:13.854431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e2165-3884-4915-8bd4-6faffac60b45 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:13.854978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e2165-3884-4915-8bd4-6faffac60b45 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:13.854978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e2165-3884-4915-8bd4-6faffac60b45 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:13.856283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 670/1341] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:13.861561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b93472-1fb3-483f-9ae0-5aafb211d3ad None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:13.862072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b93472-1fb3-483f-9ae0-5aafb211d3ad None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:13.862265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b93472-1fb3-483f-9ae0-5aafb211d3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:13.939346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-08b93472-1fb3-483f-9ae0-5aafb211d3ad None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:13.940579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-08b93472-1fb3-483f-9ae0-5aafb211d3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:14.032123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:14.050547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 672/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:57:14.089617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e2165-3884-4915-8bd4-6faffac60b45 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:14.091824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c86e2165-3884-4915-8bd4-6faffac60b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:14.191991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e2165-3884-4915-8bd4-6faffac60b45 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:14.195473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba7f11-9ab6-4716-93fd-296583993cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:14.196121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba7f11-9ab6-4716-93fd-296583993cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:14.196412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba7f11-9ab6-4716-93fd-296583993cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:14.197113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba7f11-9ab6-4716-93fd-296583993cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:14.208246 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba7f11-9ab6-4716-93fd-296583993cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:14.209728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba7f11-9ab6-4716-93fd-296583993cd8 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:14.219804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 671/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:17.107153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e1cf-c0d9-42a0-adc1-5d2ee5fc25d3 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:17.107668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e1cf-c0d9-42a0-adc1-5d2ee5fc25d3 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:17.107668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e1cf-c0d9-42a0-adc1-5d2ee5fc25d3 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:17.109024 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 673/1344] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:17.115248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9d71a-f139-4c95-b539-20d5ba3b5ba7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:17.115676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9d71a-f139-4c95-b539-20d5ba3b5ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:17.115781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9d71a-f139-4c95-b539-20d5ba3b5ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:17.143420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6cd9d71a-f139-4c95-b539-20d5ba3b5ba7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:17.144646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6cd9d71a-f139-4c95-b539-20d5ba3b5ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:17.173190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:17.222394 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 672/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:57:17.259890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e1cf-c0d9-42a0-adc1-5d2ee5fc25d3 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:17.270701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e1cf-c0d9-42a0-adc1-5d2ee5fc25d3 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:17.274230 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1df34b-5a9c-4d49-ab10-ff4d5d20ee34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:17.274869 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1df34b-5a9c-4d49-ab10-ff4d5d20ee34 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:17.275154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1df34b-5a9c-4d49-ab10-ff4d5d20ee34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:17.275432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1df34b-5a9c-4d49-ab10-ff4d5d20ee34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:17.284708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1df34b-5a9c-4d49-ab10-ff4d5d20ee34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:17.286154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1df34b-5a9c-4d49-ab10-ff4d5d20ee34 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:17.295970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 674/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:21.563044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e966db8-8ca0-4fe3-9202-4a790bf51408 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:21.563573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e966db8-8ca0-4fe3-9202-4a790bf51408 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:21.563663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e966db8-8ca0-4fe3-9202-4a790bf51408 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:21.565008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 673/1347] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:21.570217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac973c68-676d-4c57-9c82-6823c29a4ac3 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:21.570608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac973c68-676d-4c57-9c82-6823c29a4ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:21.570864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac973c68-676d-4c57-9c82-6823c29a4ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:21.589151 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ac973c68-676d-4c57-9c82-6823c29a4ac3 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:21.590339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ac973c68-676d-4c57-9c82-6823c29a4ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:21.599100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 675/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:21.638476 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e966db8-8ca0-4fe3-9202-4a790bf51408 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:21.650037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e966db8-8ca0-4fe3-9202-4a790bf51408 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:21.653896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f4ccf-b619-4319-9d6e-ff3e5a25a95e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:21.654617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4ccf-b619-4319-9d6e-ff3e5a25a95e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:21.654964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4ccf-b619-4319-9d6e-ff3e5a25a95e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:21.655307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4ccf-b619-4319-9d6e-ff3e5a25a95e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:21.665593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f4ccf-b619-4319-9d6e-ff3e5a25a95e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:21.667830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f4ccf-b619-4319-9d6e-ff3e5a25a95e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:21.679649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 674/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:27.165058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60719a3b-5670-4869-a3c3-743a0f1fb60f None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:27.165935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60719a3b-5670-4869-a3c3-743a0f1fb60f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:27.166375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60719a3b-5670-4869-a3c3-743a0f1fb60f None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:27.168157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 676/1350] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 01:57:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:57:27.172756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d616e2f-8e40-4cc0-b066-9a2980627076 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:27.173178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d616e2f-8e40-4cc0-b066-9a2980627076 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:27.173309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d616e2f-8e40-4cc0-b066-9a2980627076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:27.248516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4d616e2f-8e40-4cc0-b066-9a2980627076 None None] MFA Rules not processed for user `4eb69af145154c6d85bd4b845979e6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:27.249844 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4d616e2f-8e40-4cc0-b066-9a2980627076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:27.351551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 675/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 01:57:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:31.584133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=13 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:31.801814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f862ec-9a59-42f3-8fe4-a0885ab83e77 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:31.802461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f862ec-9a59-42f3-8fe4-a0885ab83e77 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:31.802912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f862ec-9a59-42f3-8fe4-a0885ab83e77 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:31.804569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 677/1352] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 01:57:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:31.809564 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731fe4-11f6-4902-af9e-a96d30d6a5bc None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:31.810028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731fe4-11f6-4902-af9e-a96d30d6a5bc None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:31.810349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731fe4-11f6-4902-af9e-a96d30d6a5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:31.831382 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-06731fe4-11f6-4902-af9e-a96d30d6a5bc None None] MFA Rules not processed for user `4eb69af145154c6d85bd4b845979e6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:31.832775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-06731fe4-11f6-4902-af9e-a96d30d6a5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:31.841785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 676/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 01:57:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:33.465284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35409b-add3-41f8-804a-4fba4c916bbb None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:33.465840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35409b-add3-41f8-804a-4fba4c916bbb None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:33.465979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35409b-add3-41f8-804a-4fba4c916bbb None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:33.467422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 678/1354] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 01:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:33.473767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a5d96-86ac-466b-bffe-576f8a6fd431 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:33.474312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a5d96-86ac-466b-bffe-576f8a6fd431 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:33.474402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a5d96-86ac-466b-bffe-576f8a6fd431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:33.493319 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-054a5d96-86ac-466b-bffe-576f8a6fd431 None None] MFA Rules not processed for user `4eb69af145154c6d85bd4b845979e6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:33.494508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-054a5d96-86ac-466b-bffe-576f8a6fd431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:33.503294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 677/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 01:57:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:57:37.283247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d009f4-9d12-4b60-a3bc-e1d6d075a105 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:37.283842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d009f4-9d12-4b60-a3bc-e1d6d075a105 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:37.283842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d009f4-9d12-4b60-a3bc-e1d6d075a105 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:37.285180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 679/1356] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:37.290609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676facfb-fdfc-4bb1-a4e2-ff454833b7a3 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:37.291025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676facfb-fdfc-4bb1-a4e2-ff454833b7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:37.291169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676facfb-fdfc-4bb1-a4e2-ff454833b7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:37.309153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-676facfb-fdfc-4bb1-a4e2-ff454833b7a3 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:37.310354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-676facfb-fdfc-4bb1-a4e2-ff454833b7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:37.319030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 678/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:37.386102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b112b-b28d-4a3e-a7ab-4fbd1290cddd None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:37.387311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b112b-b28d-4a3e-a7ab-4fbd1290cddd None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:37.387311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b112b-b28d-4a3e-a7ab-4fbd1290cddd None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:37.391227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 680/1358] 10.222.0.20 () {54 vars in 961 bytes} [Wed Jun 8 01:57:37 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:37.398374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16803399-6484-4e20-bdb1-15dbfb12a5ed None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:37.398993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16803399-6484-4e20-bdb1-15dbfb12a5ed None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:37.399507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16803399-6484-4e20-bdb1-15dbfb12a5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:37.451977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:37.461623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-16803399-6484-4e20-bdb1-15dbfb12a5ed None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:37.462798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-16803399-6484-4e20-bdb1-15dbfb12a5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:37.548772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 679/1359] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Jun 8 01:57:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:37.556222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b112b-b28d-4a3e-a7ab-4fbd1290cddd None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:37.572569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b112b-b28d-4a3e-a7ab-4fbd1290cddd None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:37.577924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659792-8a10-451b-b04d-70cd59135d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:37.578960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659792-8a10-451b-b04d-70cd59135d77 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:37.579468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659792-8a10-451b-b04d-70cd59135d77 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:37.579966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659792-8a10-451b-b04d-70cd59135d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:37.590783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57659792-8a10-451b-b04d-70cd59135d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:37.592554 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57659792-8a10-451b-b04d-70cd59135d77 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:37.603116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 681/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 01:57:40.265705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86723cea-31fd-4efe-be24-f3a269af4ff8 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:40.266519 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86723cea-31fd-4efe-be24-f3a269af4ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:40.266519 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86723cea-31fd-4efe-be24-f3a269af4ff8 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:40.267771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 680/1361] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:40.273076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e107c-e53f-4d00-a067-306643a51d16 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:40.273504 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e107c-e53f-4d00-a067-306643a51d16 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:40.273706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e107c-e53f-4d00-a067-306643a51d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:40.292641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b84e107c-e53f-4d00-a067-306643a51d16 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:40.293802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b84e107c-e53f-4d00-a067-306643a51d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:40.302606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 682/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:40.348816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86723cea-31fd-4efe-be24-f3a269af4ff8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:40.364555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86723cea-31fd-4efe-be24-f3a269af4ff8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:40.370140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8ea08-6759-4879-8471-9cdd200613f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:40.370890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8ea08-6759-4879-8471-9cdd200613f6 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:40.371243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8ea08-6759-4879-8471-9cdd200613f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:40.371592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8ea08-6759-4879-8471-9cdd200613f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:40.380844 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8ea08-6759-4879-8471-9cdd200613f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:40.382598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8ea08-6759-4879-8471-9cdd200613f6 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:40.393633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 681/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:42.983374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0166a-96d2-47d9-90a5-f8cbe160254a None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:42.983891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0166a-96d2-47d9-90a5-f8cbe160254a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:42.983891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0166a-96d2-47d9-90a5-f8cbe160254a None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:42.985666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 683/1364] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:42.990954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd2a4-2a4e-4a44-9f2b-742c2a955fbe None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:42.991365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd2a4-2a4e-4a44-9f2b-742c2a955fbe None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:42.991504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cd2a4-2a4e-4a44-9f2b-742c2a955fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:43.009914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3a9cd2a4-2a4e-4a44-9f2b-742c2a955fbe None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:43.011106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3a9cd2a4-2a4e-4a44-9f2b-742c2a955fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:43.019927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 682/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:43.065783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576e0bd-c55f-43dd-8d01-e5d85ef4dd3c None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:43.066080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576e0bd-c55f-43dd-8d01-e5d85ef4dd3c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:43.066358 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576e0bd-c55f-43dd-8d01-e5d85ef4dd3c None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:43.067710 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 684/1366] 10.222.0.20 () {54 vars in 961 bytes} [Wed Jun 8 01:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:43.073067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d5a41-4a46-4417-9054-a0c05b6abf44 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:43.073350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d5a41-4a46-4417-9054-a0c05b6abf44 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:43.073621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d5a41-4a46-4417-9054-a0c05b6abf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:43.091883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0e8d5a41-4a46-4417-9054-a0c05b6abf44 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:43.093100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0e8d5a41-4a46-4417-9054-a0c05b6abf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:43.102225 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 683/1367] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Jun 8 01:57:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:43.110142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576e0bd-c55f-43dd-8d01-e5d85ef4dd3c None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:43.124318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576e0bd-c55f-43dd-8d01-e5d85ef4dd3c None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:43.130381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1fa615-e17a-408f-86a0-d287bb132332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:43.131103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fa615-e17a-408f-86a0-d287bb132332 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:43.131458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fa615-e17a-408f-86a0-d287bb132332 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:43.131804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fa615-e17a-408f-86a0-d287bb132332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:43.139969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1fa615-e17a-408f-86a0-d287bb132332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:43.141634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1fa615-e17a-408f-86a0-d287bb132332 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:43.151297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 685/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:45.517670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d01d2a-c3af-4dd7-8677-1139aa6371fb None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:45.518270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d01d2a-c3af-4dd7-8677-1139aa6371fb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:45.518270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d01d2a-c3af-4dd7-8677-1139aa6371fb None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:45.519586 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 684/1369] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:45.525147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62f7b5-cdd6-4a35-9dd7-04f866ff5d48 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:45.525562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62f7b5-cdd6-4a35-9dd7-04f866ff5d48 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:45.525703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62f7b5-cdd6-4a35-9dd7-04f866ff5d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:45.547301 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7b62f7b5-cdd6-4a35-9dd7-04f866ff5d48 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:45.548914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7b62f7b5-cdd6-4a35-9dd7-04f866ff5d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:45.557231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 686/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:45.602952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d01d2a-c3af-4dd7-8677-1139aa6371fb None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:45.617899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d01d2a-c3af-4dd7-8677-1139aa6371fb None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:45.623628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5413d1-7807-4d1f-b489-7705723da79e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:45.624359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5413d1-7807-4d1f-b489-7705723da79e service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:45.624798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5413d1-7807-4d1f-b489-7705723da79e service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:45.625144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5413d1-7807-4d1f-b489-7705723da79e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:45.635436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5413d1-7807-4d1f-b489-7705723da79e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:45.637221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5413d1-7807-4d1f-b489-7705723da79e service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:45.649569 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 685/1371] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:45.711627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a291c4a-fbfd-435f-8132-1302d05849b9 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:45.711920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a291c4a-fbfd-435f-8132-1302d05849b9 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:45.712656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a291c4a-fbfd-435f-8132-1302d05849b9 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:45.714376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 687/1372] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:45.721981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d5f27-dcb4-4c10-b42a-a71195784162 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:45.722606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d5f27-dcb4-4c10-b42a-a71195784162 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:45.723073 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d5f27-dcb4-4c10-b42a-a71195784162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:45.751259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7c0d5f27-dcb4-4c10-b42a-a71195784162 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:45.752768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7c0d5f27-dcb4-4c10-b42a-a71195784162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:45.764446 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 686/1373] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:46.040427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa8dba-ecc3-46f2-8a9e-9709017afc9b None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:46.040808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa8dba-ecc3-46f2-8a9e-9709017afc9b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:46.041098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa8dba-ecc3-46f2-8a9e-9709017afc9b None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:46.042440 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 688/1374] 10.222.0.20 () {54 vars in 963 bytes} [Wed Jun 8 01:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:46.047596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4d7e3-41ba-48d3-8938-7aa5647a5b21 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:46.048100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4d7e3-41ba-48d3-8938-7aa5647a5b21 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:46.048584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4d7e3-41ba-48d3-8938-7aa5647a5b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:46.127663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fad4d7e3-41ba-48d3-8938-7aa5647a5b21 None None] MFA Rules not processed for user `236226ec865b49febc7713df1c0fab98`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:46.128919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fad4d7e3-41ba-48d3-8938-7aa5647a5b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:46.204468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 687/1375] 10.222.0.20 () {58 vars in 1060 bytes} [Wed Jun 8 01:57:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:46.214071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa8dba-ecc3-46f2-8a9e-9709017afc9b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:46.225912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa8dba-ecc3-46f2-8a9e-9709017afc9b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:46.230196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6a537-619b-4757-acd4-c9a7eb813d05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:46.230841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a537-619b-4757-acd4-c9a7eb813d05 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:46.231125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a537-619b-4757-acd4-c9a7eb813d05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:46.231401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a537-619b-4757-acd4-c9a7eb813d05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:46.241751 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6a537-619b-4757-acd4-c9a7eb813d05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:46.243202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6a537-619b-4757-acd4-c9a7eb813d05 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:46.256808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 689/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:47.973692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d52df7-16f2-4e6e-a3dd-dcd368add6e6 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:47.974240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d52df7-16f2-4e6e-a3dd-dcd368add6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:47.974240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d52df7-16f2-4e6e-a3dd-dcd368add6e6 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:47.975617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 688/1377] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:47.980984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8716006-72dc-401b-b3e2-265ac4971bfa None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:47.981430 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8716006-72dc-401b-b3e2-265ac4971bfa None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:47.981531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8716006-72dc-401b-b3e2-265ac4971bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:47.999287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e8716006-72dc-401b-b3e2-265ac4971bfa None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:48.000609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e8716006-72dc-401b-b3e2-265ac4971bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:48.008643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 690/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:57:48.055124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d52df7-16f2-4e6e-a3dd-dcd368add6e6 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:48.069899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d52df7-16f2-4e6e-a3dd-dcd368add6e6 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:48.075512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12816bda-6546-4ca4-8535-49234e4aa96f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:48.076195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12816bda-6546-4ca4-8535-49234e4aa96f service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:48.076504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12816bda-6546-4ca4-8535-49234e4aa96f service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:48.077186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12816bda-6546-4ca4-8535-49234e4aa96f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:48.088060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12816bda-6546-4ca4-8535-49234e4aa96f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:48.089920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12816bda-6546-4ca4-8535-49234e4aa96f service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:48.095174 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:57:48.103155 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 689/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:50.424580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce8f32-6a11-46e5-9577-7962a572580a None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:50.425134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce8f32-6a11-46e5-9577-7962a572580a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:50.425440 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce8f32-6a11-46e5-9577-7962a572580a None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:50.427187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 691/1380] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:50.432637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566b7cc-cddd-4e67-8b7a-35bdc4320a1b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:50.433067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566b7cc-cddd-4e67-8b7a-35bdc4320a1b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:50.433353 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566b7cc-cddd-4e67-8b7a-35bdc4320a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:50.452560 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b566b7cc-cddd-4e67-8b7a-35bdc4320a1b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:50.454755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b566b7cc-cddd-4e67-8b7a-35bdc4320a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:50.463221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 690/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:50.507538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce8f32-6a11-46e5-9577-7962a572580a None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:50.522520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce8f32-6a11-46e5-9577-7962a572580a None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:50.528190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa4b23-9ed2-4d43-b612-0bce9d35dc65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:50.528987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa4b23-9ed2-4d43-b612-0bce9d35dc65 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:50.529382 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa4b23-9ed2-4d43-b612-0bce9d35dc65 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:50.529773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa4b23-9ed2-4d43-b612-0bce9d35dc65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:50.539446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa4b23-9ed2-4d43-b612-0bce9d35dc65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:50.541197 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa4b23-9ed2-4d43-b612-0bce9d35dc65 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:50.551955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 692/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:52.880483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4761a5-852d-42c1-a628-e5771067cb72 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:52.881478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4761a5-852d-42c1-a628-e5771067cb72 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:52.882004 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4761a5-852d-42c1-a628-e5771067cb72 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:52.883993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 691/1383] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:52.891420 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916892f-8fda-4654-b7f9-ad1281caf898 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:52.892012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916892f-8fda-4654-b7f9-ad1281caf898 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:52.892476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916892f-8fda-4654-b7f9-ad1281caf898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:52.915960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-2916892f-8fda-4654-b7f9-ad1281caf898 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:52.917516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2916892f-8fda-4654-b7f9-ad1281caf898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:52.927232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 693/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:57:52.969462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4761a5-852d-42c1-a628-e5771067cb72 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:52.984916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4761a5-852d-42c1-a628-e5771067cb72 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:52.990701 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77531c56-c802-48ce-bbc1-b8f4a2caef32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:52.991453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77531c56-c802-48ce-bbc1-b8f4a2caef32 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:52.991845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77531c56-c802-48ce-bbc1-b8f4a2caef32 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:52.992249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77531c56-c802-48ce-bbc1-b8f4a2caef32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:53.001318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77531c56-c802-48ce-bbc1-b8f4a2caef32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:53.003017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77531c56-c802-48ce-bbc1-b8f4a2caef32 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:53.013427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 692/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 01:57:55.275917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2efb3b-4824-438a-8bfe-df6f3f6225da None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:55.276496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2efb3b-4824-438a-8bfe-df6f3f6225da None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:55.276496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2efb3b-4824-438a-8bfe-df6f3f6225da None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:55.277855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 694/1386] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:55.283324 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052b235-c76e-4ff5-a749-71788976cbe4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:55.284127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052b235-c76e-4ff5-a749-71788976cbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:55.284438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052b235-c76e-4ff5-a749-71788976cbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:55.303121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7052b235-c76e-4ff5-a749-71788976cbe4 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:55.304306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7052b235-c76e-4ff5-a749-71788976cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:55.312030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 693/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:57:55.357342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2efb3b-4824-438a-8bfe-df6f3f6225da None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:55.377791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2efb3b-4824-438a-8bfe-df6f3f6225da None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:55.384704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc2ab5-2cd0-4733-b442-12862d4ae4ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:55.385731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc2ab5-2cd0-4733-b442-12862d4ae4ac service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:55.386370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc2ab5-2cd0-4733-b442-12862d4ae4ac service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:55.387011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc2ab5-2cd0-4733-b442-12862d4ae4ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:55.398191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc2ab5-2cd0-4733-b442-12862d4ae4ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:55.400282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc2ab5-2cd0-4733-b442-12862d4ae4ac service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:55.413892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 695/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:57:58.501853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bcbae-55ee-4d56-889e-70ef29328d4c None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:58.502665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bcbae-55ee-4d56-889e-70ef29328d4c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:58.502665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bcbae-55ee-4d56-889e-70ef29328d4c None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:58.503876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 694/1389] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:57:58.509306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da91d68-32a6-4005-ba17-ee559a8fa700 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:58.509764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da91d68-32a6-4005-ba17-ee559a8fa700 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:58.509917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da91d68-32a6-4005-ba17-ee559a8fa700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:58.529066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3da91d68-32a6-4005-ba17-ee559a8fa700 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:57:58.530272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3da91d68-32a6-4005-ba17-ee559a8fa700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:57:58.539135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 696/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:57:58.594726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bcbae-55ee-4d56-889e-70ef29328d4c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:57:58.609405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bcbae-55ee-4d56-889e-70ef29328d4c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:57:58.614893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe32e1-e543-4cdc-9b36-f531d0316ac9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:57:58.615536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe32e1-e543-4cdc-9b36-f531d0316ac9 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:57:58.615819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe32e1-e543-4cdc-9b36-f531d0316ac9 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:57:58.616153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe32e1-e543-4cdc-9b36-f531d0316ac9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:57:58.624579 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe32e1-e543-4cdc-9b36-f531d0316ac9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:57:58.626275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe32e1-e543-4cdc-9b36-f531d0316ac9 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:57:58.637777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 695/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 01:58:01.099074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45673dcc-6cba-47fc-b79c-9696930d6638 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:01.099648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45673dcc-6cba-47fc-b79c-9696930d6638 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:01.099648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45673dcc-6cba-47fc-b79c-9696930d6638 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:01.101114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 697/1392] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:01.106263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcde7b-47b9-466a-8339-b4bd990b2d73 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:01.106732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcde7b-47b9-466a-8339-b4bd990b2d73 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:01.106833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcde7b-47b9-466a-8339-b4bd990b2d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:01.124815 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4fdcde7b-47b9-466a-8339-b4bd990b2d73 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:01.126001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4fdcde7b-47b9-466a-8339-b4bd990b2d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:01.133823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 696/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:01.178600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45673dcc-6cba-47fc-b79c-9696930d6638 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:01.193055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45673dcc-6cba-47fc-b79c-9696930d6638 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:01.198445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7fc50-94f3-4298-a666-ced863eea503 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:01.199088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7fc50-94f3-4298-a666-ced863eea503 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:01.199370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7fc50-94f3-4298-a666-ced863eea503 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:01.199645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7fc50-94f3-4298-a666-ced863eea503 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:01.211203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7fc50-94f3-4298-a666-ced863eea503 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:01.213181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7fc50-94f3-4298-a666-ced863eea503 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:01.225983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 698/1394] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:02.201606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e8685-cd01-46cd-ad53-0003898a4416 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:02.202210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e8685-cd01-46cd-ad53-0003898a4416 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:02.202210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e8685-cd01-46cd-ad53-0003898a4416 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:02.204308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 697/1395] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:02.209776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60971b-1d03-469d-8526-fbfcc373ac92 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:02.210220 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60971b-1d03-469d-8526-fbfcc373ac92 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:02.210416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60971b-1d03-469d-8526-fbfcc373ac92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:02.216565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=52 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:58:02.229885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-cf60971b-1d03-469d-8526-fbfcc373ac92 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:02.231071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-cf60971b-1d03-469d-8526-fbfcc373ac92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:02.239239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 699/1396] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:58:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:02.295321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b8bfa-b400-4bc6-9c94-66d6d15d7398 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:02.295608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b8bfa-b400-4bc6-9c94-66d6d15d7398 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:02.295891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b8bfa-b400-4bc6-9c94-66d6d15d7398 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:02.297335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 698/1397] 10.222.0.20 () {54 vars in 962 bytes} [Wed Jun 8 01:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:02.302491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f23ef-d885-4e15-8a71-1c94dc734240 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:02.302840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f23ef-d885-4e15-8a71-1c94dc734240 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:02.303180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f23ef-d885-4e15-8a71-1c94dc734240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:02.352051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-152f23ef-d885-4e15-8a71-1c94dc734240 None None] MFA Rules not processed for user `7d1e01f6f82a4ebbb15d0cb55b963af4`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:02.353198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-152f23ef-d885-4e15-8a71-1c94dc734240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:02.415321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 700/1398] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Jun 8 01:58:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:02.423188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b8bfa-b400-4bc6-9c94-66d6d15d7398 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:02.433770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b8bfa-b400-4bc6-9c94-66d6d15d7398 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:02.438018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190e10b-f38f-4e3e-a44c-ccd92e3ea414 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d1e01f6f82a4ebbb15d0cb55b963af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:02.438657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190e10b-f38f-4e3e-a44c-ccd92e3ea414 service glance] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:02.438937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190e10b-f38f-4e3e-a44c-ccd92e3ea414 service glance] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:02.439209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190e10b-f38f-4e3e-a44c-ccd92e3ea414 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:02.448219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190e10b-f38f-4e3e-a44c-ccd92e3ea414 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:02.449709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190e10b-f38f-4e3e-a44c-ccd92e3ea414 service glance] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:02.459480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 699/1399] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:02.996518 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ede63-986a-44f5-9b50-233c19bc4318 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:02.996879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ede63-986a-44f5-9b50-233c19bc4318 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:02.997225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ede63-986a-44f5-9b50-233c19bc4318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:03.045808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-297ede63-986a-44f5-9b50-233c19bc4318 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:03.047910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-297ede63-986a-44f5-9b50-233c19bc4318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:03.112608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 701/1400] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 01:58:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:03.157692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7b8fe-3485-41c7-8566-3607e11920d5 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:03.157984 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7b8fe-3485-41c7-8566-3607e11920d5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:03.158262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7b8fe-3485-41c7-8566-3607e11920d5 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:03.159648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 700/1401] 10.222.0.20 () {54 vars in 942 bytes} [Wed Jun 8 01:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:03.166099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a1c43-4305-47dd-9c26-9570c8da80d6 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:03.166393 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a1c43-4305-47dd-9c26-9570c8da80d6 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:03.166671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a1c43-4305-47dd-9c26-9570c8da80d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:03.221054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-a84a1c43-4305-47dd-9c26-9570c8da80d6 None None] MFA Rules not processed for user `fc299258fe814636880cf595fc2904b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:03.222283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a84a1c43-4305-47dd-9c26-9570c8da80d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:03.236501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:58:03.297182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 702/1402] 10.222.0.20 () {58 vars in 1039 bytes} [Wed Jun 8 01:58:03 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:58:03.306449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7b8fe-3485-41c7-8566-3607e11920d5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:03.319973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7b8fe-3485-41c7-8566-3607e11920d5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:03.324979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3593a13-af5e-48b1-9e09-c557c4cab237 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:03.325641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3593a13-af5e-48b1-9e09-c557c4cab237 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:03.325916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3593a13-af5e-48b1-9e09-c557c4cab237 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:03.326187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3593a13-af5e-48b1-9e09-c557c4cab237 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:03.334596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3593a13-af5e-48b1-9e09-c557c4cab237 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:03.336163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3593a13-af5e-48b1-9e09-c557c4cab237 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:03.345326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 701/1403] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 01:58:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 01:58:03.544658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41334268-e1ac-497d-923c-db5aaca9ee6c None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:03.545305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41334268-e1ac-497d-923c-db5aaca9ee6c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:03.545645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41334268-e1ac-497d-923c-db5aaca9ee6c None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:03.547396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 703/1404] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:58:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:03.553029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365194af-de2e-43cb-8795-d8edaabecea4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:03.553336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365194af-de2e-43cb-8795-d8edaabecea4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:03.553644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365194af-de2e-43cb-8795-d8edaabecea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:03.572335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-365194af-de2e-43cb-8795-d8edaabecea4 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:03.573597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-365194af-de2e-43cb-8795-d8edaabecea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:03.586618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 702/1405] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:03.631162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41334268-e1ac-497d-923c-db5aaca9ee6c None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:03.646891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41334268-e1ac-497d-923c-db5aaca9ee6c None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:03.655203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc8098-753d-4494-9634-6db83331c4cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:03.655957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc8098-753d-4494-9634-6db83331c4cf service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:03.656287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc8098-753d-4494-9634-6db83331c4cf service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:03.656652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc8098-753d-4494-9634-6db83331c4cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:03.665910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc8098-753d-4494-9634-6db83331c4cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:03.667503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc8098-753d-4494-9634-6db83331c4cf service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:03.679419 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 704/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:06.035606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3d0d6-2499-4b86-92ba-876bbb70572e None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:06.036157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3d0d6-2499-4b86-92ba-876bbb70572e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:06.036157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3d0d6-2499-4b86-92ba-876bbb70572e None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:06.037620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 703/1407] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:06.043189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa97d79-cebf-4661-8713-335f9da2a8e8 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:06.043603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa97d79-cebf-4661-8713-335f9da2a8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:06.043762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa97d79-cebf-4661-8713-335f9da2a8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:06.063277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-0fa97d79-cebf-4661-8713-335f9da2a8e8 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:06.064503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0fa97d79-cebf-4661-8713-335f9da2a8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:06.076149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 705/1408] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:06.119466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3d0d6-2499-4b86-92ba-876bbb70572e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:06.135248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3d0d6-2499-4b86-92ba-876bbb70572e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:06.141443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44350305-3206-46bf-b4d5-979437281a62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:06.142086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350305-3206-46bf-b4d5-979437281a62 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:06.142366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350305-3206-46bf-b4d5-979437281a62 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:06.142639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350305-3206-46bf-b4d5-979437281a62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:06.151946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44350305-3206-46bf-b4d5-979437281a62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:06.153637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44350305-3206-46bf-b4d5-979437281a62 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:06.165614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 704/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:16.057870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=33 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:58:16.159169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=43 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:58:26.844661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9013aa8-2049-46c9-9bda-b304e52f6670 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:26.845259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9013aa8-2049-46c9-9bda-b304e52f6670 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:26.845259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9013aa8-2049-46c9-9bda-b304e52f6670 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:26.846564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 706/1410] 10.222.0.20 () {54 vars in 918 bytes} [Wed Jun 8 01:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:58:26.851567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529566f-6122-4329-9522-ef92ab3ee238 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:26.851956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529566f-6122-4329-9522-ef92ab3ee238 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:26.852115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529566f-6122-4329-9522-ef92ab3ee238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:26.869965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5529566f-6122-4329-9522-ef92ab3ee238 None None] MFA Rules not processed for user `4eb69af145154c6d85bd4b845979e6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:26.871154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5529566f-6122-4329-9522-ef92ab3ee238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:26.879628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 705/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Jun 8 01:58:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:26.894386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a6a91-1a53-4028-bff7-09638c8563ed None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:26.894707 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a6a91-1a53-4028-bff7-09638c8563ed None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:26.894986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a6a91-1a53-4028-bff7-09638c8563ed None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:26.896325 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 707/1412] 10.222.0.20 () {54 vars in 918 bytes} [Wed Jun 8 01:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:26.900838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62372c94-d48a-4517-9292-0d2141a45d97 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:26.901126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62372c94-d48a-4517-9292-0d2141a45d97 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:26.901404 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62372c94-d48a-4517-9292-0d2141a45d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:26.920074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-62372c94-d48a-4517-9292-0d2141a45d97 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:26.921449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-62372c94-d48a-4517-9292-0d2141a45d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:26.930497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 706/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Jun 8 01:58:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:58:29.630943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dc1cb-7ee1-444d-9de8-b099f6982617 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:29.631521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dc1cb-7ee1-444d-9de8-b099f6982617 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:29.631521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092dc1cb-7ee1-444d-9de8-b099f6982617 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:29.632880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:29.638210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13ae7-3ab1-449f-8448-24b90b9bd700 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:29.638631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13ae7-3ab1-449f-8448-24b90b9bd700 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:29.638736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13ae7-3ab1-449f-8448-24b90b9bd700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:29.657266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ceb13ae7-3ab1-449f-8448-24b90b9bd700 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:29.658480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ceb13ae7-3ab1-449f-8448-24b90b9bd700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:29.666398 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:29.711211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dc1cb-7ee1-444d-9de8-b099f6982617 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:29.726837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092dc1cb-7ee1-444d-9de8-b099f6982617 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:29.732109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7aa85-e068-414b-ade6-1bc0ff26dd49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:29.733388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7aa85-e068-414b-ade6-1bc0ff26dd49 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:29.733686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7aa85-e068-414b-ade6-1bc0ff26dd49 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:29.733974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7aa85-e068-414b-ade6-1bc0ff26dd49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:29.742002 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7aa85-e068-414b-ade6-1bc0ff26dd49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:29.743572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7aa85-e068-414b-ade6-1bc0ff26dd49 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:29.752783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 709/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:30.754483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdabf7-fd68-4583-956b-e7250d574723 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:30.755054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdabf7-fd68-4583-956b-e7250d574723 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:30.755054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdabf7-fd68-4583-956b-e7250d574723 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:30.756468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 01:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:30.761571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb70223-ac09-4373-82ff-7f76f998a375 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:30.762015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb70223-ac09-4373-82ff-7f76f998a375 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:30.762113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb70223-ac09-4373-82ff-7f76f998a375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:30.781896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-0cb70223-ac09-4373-82ff-7f76f998a375 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:30.783076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0cb70223-ac09-4373-82ff-7f76f998a375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:30.790900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 01:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:30.958219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdabf7-fd68-4583-956b-e7250d574723 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:30.972479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdabf7-fd68-4583-956b-e7250d574723 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:30.977833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799839a-8ffc-4f7c-a899-82800fa0bbe7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:30.978467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799839a-8ffc-4f7c-a899-82800fa0bbe7 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:30.978753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799839a-8ffc-4f7c-a899-82800fa0bbe7 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:30.979032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799839a-8ffc-4f7c-a899-82800fa0bbe7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:30.987128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799839a-8ffc-4f7c-a899-82800fa0bbe7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:30.988762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799839a-8ffc-4f7c-a899-82800fa0bbe7 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:30.997990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 709/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:31.929919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5feb654-2a6c-4ff9-9b98-e30b2324b159 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:31.930426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5feb654-2a6c-4ff9-9b98-e30b2324b159 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:31.930539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5feb654-2a6c-4ff9-9b98-e30b2324b159 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:31.931813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 01:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:31.936969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e6344-c1ec-4697-8648-f422ceb2e339 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:31.937378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e6344-c1ec-4697-8648-f422ceb2e339 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:31.937525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e6344-c1ec-4697-8648-f422ceb2e339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:31.955266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ba0e6344-c1ec-4697-8648-f422ceb2e339 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:31.956490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ba0e6344-c1ec-4697-8648-f422ceb2e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:31.964169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 01:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:32.113922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5feb654-2a6c-4ff9-9b98-e30b2324b159 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:32.128671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5feb654-2a6c-4ff9-9b98-e30b2324b159 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:32.133971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc0d0a-84d7-4d96-9250-a56399fa91ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:32.134601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc0d0a-84d7-4d96-9250-a56399fa91ab service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:32.134885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc0d0a-84d7-4d96-9250-a56399fa91ab service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:32.135161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc0d0a-84d7-4d96-9250-a56399fa91ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:32.143162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc0d0a-84d7-4d96-9250-a56399fa91ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:32.144794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc0d0a-84d7-4d96-9250-a56399fa91ab service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:32.154078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 712/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:33.071803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9177f-7842-432c-997a-103b54c6ac37 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:33.072419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9177f-7842-432c-997a-103b54c6ac37 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:33.072419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9177f-7842-432c-997a-103b54c6ac37 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:33.073754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 01:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:33.079256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba4e92-9563-4905-a7f0-fc28b7888076 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:33.079696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba4e92-9563-4905-a7f0-fc28b7888076 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:33.079802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba4e92-9563-4905-a7f0-fc28b7888076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:33.097882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-20ba4e92-9563-4905-a7f0-fc28b7888076 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:33.099051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-20ba4e92-9563-4905-a7f0-fc28b7888076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:33.106998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 01:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:33.258275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9177f-7842-432c-997a-103b54c6ac37 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:33.273142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9177f-7842-432c-997a-103b54c6ac37 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:33.278403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dab00f-32b7-450e-8c85-8320fa1974cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:33.279032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dab00f-32b7-450e-8c85-8320fa1974cd service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:33.279314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dab00f-32b7-450e-8c85-8320fa1974cd service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:33.279592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dab00f-32b7-450e-8c85-8320fa1974cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:33.287568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dab00f-32b7-450e-8c85-8320fa1974cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:33.289441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dab00f-32b7-450e-8c85-8320fa1974cd service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:33.300106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 712/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:34.217724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34a76f-b6e4-4c39-9242-1536076c9852 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:34.218378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34a76f-b6e4-4c39-9242-1536076c9852 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:34.218378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34a76f-b6e4-4c39-9242-1536076c9852 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:34.219537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 01:58:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:34.224580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52883-b0da-4dce-bcea-47431862c5ea None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:34.225018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52883-b0da-4dce-bcea-47431862c5ea None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:34.225191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52883-b0da-4dce-bcea-47431862c5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:34.243378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1cd52883-b0da-4dce-bcea-47431862c5ea None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:34.244611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1cd52883-b0da-4dce-bcea-47431862c5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:34.252441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 01:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:34.400941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34a76f-b6e4-4c39-9242-1536076c9852 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:34.416514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34a76f-b6e4-4c39-9242-1536076c9852 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:34.421988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6b8e5-582f-478e-8144-9d25d17693ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:34.422613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b8e5-582f-478e-8144-9d25d17693ba service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:34.422892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b8e5-582f-478e-8144-9d25d17693ba service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:34.423163 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b8e5-582f-478e-8144-9d25d17693ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:34.431066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b8e5-582f-478e-8144-9d25d17693ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:34.432664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b8e5-582f-478e-8144-9d25d17693ba service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:34.442802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 715/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:35.288173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98153fe7-a079-4721-864b-966cbc18b7fc None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:35.288990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98153fe7-a079-4721-864b-966cbc18b7fc None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:35.289087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98153fe7-a079-4721-864b-966cbc18b7fc None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:35.290413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 01:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:35.296137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80c9fd-7adc-41c5-aaa2-d32228e2dd61 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:35.296634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80c9fd-7adc-41c5-aaa2-d32228e2dd61 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:35.296792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80c9fd-7adc-41c5-aaa2-d32228e2dd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:35.314964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-2f80c9fd-7adc-41c5-aaa2-d32228e2dd61 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:35.316142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2f80c9fd-7adc-41c5-aaa2-d32228e2dd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:35.323767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 01:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:35.475285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98153fe7-a079-4721-864b-966cbc18b7fc None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:35.490113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98153fe7-a079-4721-864b-966cbc18b7fc None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:35.495419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57bb5b-2e24-4c4c-adb0-5ae3965618e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:35.496050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57bb5b-2e24-4c4c-adb0-5ae3965618e3 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:35.496333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57bb5b-2e24-4c4c-adb0-5ae3965618e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:35.496637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57bb5b-2e24-4c4c-adb0-5ae3965618e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:35.504652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57bb5b-2e24-4c4c-adb0-5ae3965618e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:35.506217 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57bb5b-2e24-4c4c-adb0-5ae3965618e3 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:35.515468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 715/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:36.361706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a70501-6543-4607-a430-7831969c8981 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:36.362249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a70501-6543-4607-a430-7831969c8981 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:36.362249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a70501-6543-4607-a430-7831969c8981 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:36.363593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 01:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:36.368611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371190f6-aebe-401c-a6c8-4d96a6b2840f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:36.369038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371190f6-aebe-401c-a6c8-4d96a6b2840f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:36.369200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371190f6-aebe-401c-a6c8-4d96a6b2840f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:36.387450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-371190f6-aebe-401c-a6c8-4d96a6b2840f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:36.389384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-371190f6-aebe-401c-a6c8-4d96a6b2840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:36.397182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 01:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:36.545827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a70501-6543-4607-a430-7831969c8981 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:36.560074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a70501-6543-4607-a430-7831969c8981 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:36.567836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77b308-9706-413f-b1d6-446369849e2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:36.568533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77b308-9706-413f-b1d6-446369849e2d service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:36.568819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77b308-9706-413f-b1d6-446369849e2d service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:36.569091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77b308-9706-413f-b1d6-446369849e2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:36.579211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77b308-9706-413f-b1d6-446369849e2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:36.580993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77b308-9706-413f-b1d6-446369849e2d service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:36.593416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 718/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:37.446493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bdc42-dd4c-4ed6-912f-dfcd33121c5f None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:37.447086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bdc42-dd4c-4ed6-912f-dfcd33121c5f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:37.447086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bdc42-dd4c-4ed6-912f-dfcd33121c5f None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:37.448440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 01:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:37.454014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9663e7c-6ff5-4aa6-88c8-a0bcc255e3e2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:37.454423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9663e7c-6ff5-4aa6-88c8-a0bcc255e3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:37.454578 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9663e7c-6ff5-4aa6-88c8-a0bcc255e3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:37.472980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e9663e7c-6ff5-4aa6-88c8-a0bcc255e3e2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:37.474170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e9663e7c-6ff5-4aa6-88c8-a0bcc255e3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:37.481879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 01:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:58:37.629136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bdc42-dd4c-4ed6-912f-dfcd33121c5f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:37.643998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bdc42-dd4c-4ed6-912f-dfcd33121c5f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:37.649321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76618032-049c-4c97-88af-c167ba51e0ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:37.649957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76618032-049c-4c97-88af-c167ba51e0ab service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:37.650252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76618032-049c-4c97-88af-c167ba51e0ab service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:37.650511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76618032-049c-4c97-88af-c167ba51e0ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:37.659368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76618032-049c-4c97-88af-c167ba51e0ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:37.661008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76618032-049c-4c97-88af-c167ba51e0ab service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:37.676378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 718/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:38.525527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb971db-e781-4297-a32d-8dc9bb1040cc None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:38.526108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb971db-e781-4297-a32d-8dc9bb1040cc None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:38.526108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb971db-e781-4297-a32d-8dc9bb1040cc None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:38.527433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 01:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:38.532774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245b9b9-68fe-4b63-9045-832bcecfe657 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:38.533205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245b9b9-68fe-4b63-9045-832bcecfe657 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:38.533352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245b9b9-68fe-4b63-9045-832bcecfe657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:38.551828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8245b9b9-68fe-4b63-9045-832bcecfe657 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:38.553047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8245b9b9-68fe-4b63-9045-832bcecfe657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:38.560677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 01:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:58:38.709566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb971db-e781-4297-a32d-8dc9bb1040cc None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:38.723667 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb971db-e781-4297-a32d-8dc9bb1040cc None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:38.728953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d679e73-5850-4b32-bfb0-146c06fe210c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:38.729584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d679e73-5850-4b32-bfb0-146c06fe210c service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:38.729864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d679e73-5850-4b32-bfb0-146c06fe210c service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:38.730138 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d679e73-5850-4b32-bfb0-146c06fe210c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:38.737989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d679e73-5850-4b32-bfb0-146c06fe210c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:38.739585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d679e73-5850-4b32-bfb0-146c06fe210c service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:38.748679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 721/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:40.911087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e417519-6f8d-4981-a6ea-bf5fe4d0e888 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:40.911690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e417519-6f8d-4981-a6ea-bf5fe4d0e888 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:40.911690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e417519-6f8d-4981-a6ea-bf5fe4d0e888 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:40.913249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:58:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:40.920020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5684a4e-bcca-405f-afeb-d68c4ec8fcd4 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:40.920535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5684a4e-bcca-405f-afeb-d68c4ec8fcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:40.920648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5684a4e-bcca-405f-afeb-d68c4ec8fcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:40.942116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-a5684a4e-bcca-405f-afeb-d68c4ec8fcd4 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:40.943314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a5684a4e-bcca-405f-afeb-d68c4ec8fcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:40.953619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:40.996285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e417519-6f8d-4981-a6ea-bf5fe4d0e888 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:41.007326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e417519-6f8d-4981-a6ea-bf5fe4d0e888 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:41.010788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759c615-e490-4182-b12b-366608aab895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:41.011413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c615-e490-4182-b12b-366608aab895 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:41.011692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c615-e490-4182-b12b-366608aab895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:41.011991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c615-e490-4182-b12b-366608aab895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:41.021475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759c615-e490-4182-b12b-366608aab895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:41.022893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759c615-e490-4182-b12b-366608aab895 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:41.032371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 721/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:43.419551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb833e4-75b6-4288-89aa-4841fd17881e None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:43.420192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb833e4-75b6-4288-89aa-4841fd17881e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:43.420192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb833e4-75b6-4288-89aa-4841fd17881e None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:43.422020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:43.427459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971955d-181b-4ac9-9889-ab4f43d72fd0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:43.427864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971955d-181b-4ac9-9889-ab4f43d72fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:43.428026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971955d-181b-4ac9-9889-ab4f43d72fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:43.446864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9971955d-181b-4ac9-9889-ab4f43d72fd0 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:43.448063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9971955d-181b-4ac9-9889-ab4f43d72fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:43.455877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:43.493513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb833e4-75b6-4288-89aa-4841fd17881e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:43.505248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb833e4-75b6-4288-89aa-4841fd17881e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:43.508754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb9201-e3ec-4779-97b6-ccbbe07eff86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:43.509384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb9201-e3ec-4779-97b6-ccbbe07eff86 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:43.509665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb9201-e3ec-4779-97b6-ccbbe07eff86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:43.509938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb9201-e3ec-4779-97b6-ccbbe07eff86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:43.518674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb9201-e3ec-4779-97b6-ccbbe07eff86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:43.520100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb9201-e3ec-4779-97b6-ccbbe07eff86 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:43.529459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 724/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:46.051505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e6881-0a88-4f62-845c-7b6e844d1fcd None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:46.052069 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e6881-0a88-4f62-845c-7b6e844d1fcd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:46.052069 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e6881-0a88-4f62-845c-7b6e844d1fcd None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:46.053435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:46.058697 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66616cc6-25ce-4f86-b278-492716aab9d9 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:46.059107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66616cc6-25ce-4f86-b278-492716aab9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:46.059248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66616cc6-25ce-4f86-b278-492716aab9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:46.077978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-66616cc6-25ce-4f86-b278-492716aab9d9 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:46.079231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-66616cc6-25ce-4f86-b278-492716aab9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:46.087004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:46.125995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e6881-0a88-4f62-845c-7b6e844d1fcd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:46.136507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e6881-0a88-4f62-845c-7b6e844d1fcd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:46.140143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a12f6f-a8d0-4815-8454-31112c313842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:46.140810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a12f6f-a8d0-4815-8454-31112c313842 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:46.141094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a12f6f-a8d0-4815-8454-31112c313842 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:46.141369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a12f6f-a8d0-4815-8454-31112c313842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:46.150039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a12f6f-a8d0-4815-8454-31112c313842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:46.151471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a12f6f-a8d0-4815-8454-31112c313842 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:46.161625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 724/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:48.570136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b6a04-e175-4b58-8280-9c108537286a None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:48.570728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b6a04-e175-4b58-8280-9c108537286a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:48.570900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b6a04-e175-4b58-8280-9c108537286a None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:48.572664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:58:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:48.579048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d46e9c-970d-4ac6-9794-07fb0ff50683 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:48.579562 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d46e9c-970d-4ac6-9794-07fb0ff50683 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:48.579920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d46e9c-970d-4ac6-9794-07fb0ff50683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:48.598852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-48d46e9c-970d-4ac6-9794-07fb0ff50683 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:48.600248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-48d46e9c-970d-4ac6-9794-07fb0ff50683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:48.608620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:48.649611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b6a04-e175-4b58-8280-9c108537286a None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:48.651298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ed1b6a04-e175-4b58-8280-9c108537286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:48.665537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b6a04-e175-4b58-8280-9c108537286a None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:48.669088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65ce2f-76ae-4a34-8561-efbee0bfc86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:48.669718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65ce2f-76ae-4a34-8561-efbee0bfc86e service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:48.670002 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65ce2f-76ae-4a34-8561-efbee0bfc86e service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:48.670278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65ce2f-76ae-4a34-8561-efbee0bfc86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:48.679439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65ce2f-76ae-4a34-8561-efbee0bfc86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:48.680920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65ce2f-76ae-4a34-8561-efbee0bfc86e service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:48.690223 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 727/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:51.076297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9ffc1-fc21-495b-9bc3-60766e39845a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:51.076879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9ffc1-fc21-495b-9bc3-60766e39845a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:51.076879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9ffc1-fc21-495b-9bc3-60766e39845a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:51.078775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:58:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:51.084002 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468843d0-4e49-4448-89cb-f415dc6713b2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:51.084483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468843d0-4e49-4448-89cb-f415dc6713b2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:51.084604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468843d0-4e49-4448-89cb-f415dc6713b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:51.102478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-468843d0-4e49-4448-89cb-f415dc6713b2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:51.103684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-468843d0-4e49-4448-89cb-f415dc6713b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:51.111529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:51.151086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9ffc1-fc21-495b-9bc3-60766e39845a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:51.162182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9ffc1-fc21-495b-9bc3-60766e39845a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:51.165951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad2dbd-4a04-4486-a9a8-3aa911930c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:51.166518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2dbd-4a04-4486-a9a8-3aa911930c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:51.166796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2dbd-4a04-4486-a9a8-3aa911930c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:51.167068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2dbd-4a04-4486-a9a8-3aa911930c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:51.175743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad2dbd-4a04-4486-a9a8-3aa911930c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:51.177200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad2dbd-4a04-4486-a9a8-3aa911930c88 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:51.186436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 727/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:53.519069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92351cd-acde-4861-ba06-4b8bc004d241 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:53.519614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92351cd-acde-4861-ba06-4b8bc004d241 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:53.519614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92351cd-acde-4861-ba06-4b8bc004d241 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:53.521015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:53.526178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43c6a3-e06b-48f4-ae32-34339d4fcec7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:53.526590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43c6a3-e06b-48f4-ae32-34339d4fcec7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:53.526816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43c6a3-e06b-48f4-ae32-34339d4fcec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:53.544853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6f43c6a3-e06b-48f4-ae32-34339d4fcec7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:53.546074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6f43c6a3-e06b-48f4-ae32-34339d4fcec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:53.553935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:53.592283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92351cd-acde-4861-ba06-4b8bc004d241 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:53.602697 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92351cd-acde-4861-ba06-4b8bc004d241 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:53.606150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c90ab-59ce-4f0f-a137-bad6de19afde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:53.606768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c90ab-59ce-4f0f-a137-bad6de19afde service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:53.607045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c90ab-59ce-4f0f-a137-bad6de19afde service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:53.607324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c90ab-59ce-4f0f-a137-bad6de19afde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:53.616418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c90ab-59ce-4f0f-a137-bad6de19afde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:53.617851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c90ab-59ce-4f0f-a137-bad6de19afde service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:53.627904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 730/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:55.943427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04272e-7ad2-4430-a931-5f5b46e41be5 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:55.943949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04272e-7ad2-4430-a931-5f5b46e41be5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:55.944060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04272e-7ad2-4430-a931-5f5b46e41be5 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:55.945433 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:55.950629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cdce7-31bf-4a72-8d7d-6d2004accadc None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:55.951023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cdce7-31bf-4a72-8d7d-6d2004accadc None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:55.951186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cdce7-31bf-4a72-8d7d-6d2004accadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:55.969461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-796cdce7-31bf-4a72-8d7d-6d2004accadc None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:55.970650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-796cdce7-31bf-4a72-8d7d-6d2004accadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:55.978493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:56.016163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04272e-7ad2-4430-a931-5f5b46e41be5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:56.026786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04272e-7ad2-4430-a931-5f5b46e41be5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:56.030792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0811e69-2eb3-4631-8c29-42f97e9885a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:56.031614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0811e69-2eb3-4631-8c29-42f97e9885a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:56.031901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0811e69-2eb3-4631-8c29-42f97e9885a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:56.032181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0811e69-2eb3-4631-8c29-42f97e9885a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:56.041266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0811e69-2eb3-4631-8c29-42f97e9885a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:56.042697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0811e69-2eb3-4631-8c29-42f97e9885a4 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:56.052896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 730/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:58:58.364179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f6377-74a3-42df-911f-a2f490f56153 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:58.364787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f6377-74a3-42df-911f-a2f490f56153 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:58.364787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f6377-74a3-42df-911f-a2f490f56153 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:58.366140 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:58:58.371550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb082c-1a8d-4559-87c6-4ad60634fa36 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:58.371982 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb082c-1a8d-4559-87c6-4ad60634fa36 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:58.372097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb082c-1a8d-4559-87c6-4ad60634fa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:58.378306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=79 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:58:58.391392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3dbb082c-1a8d-4559-87c6-4ad60634fa36 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:58:58.392619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3dbb082c-1a8d-4559-87c6-4ad60634fa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:58:58.400324 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:58:58.438240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f6377-74a3-42df-911f-a2f490f56153 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:58:58.443495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=73 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:58:58.449990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f6377-74a3-42df-911f-a2f490f56153 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:58:58.454454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8737b9-2b88-4b2c-b648-36e0e52b76f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:58:58.455081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8737b9-2b88-4b2c-b648-36e0e52b76f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:58:58.455363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8737b9-2b88-4b2c-b648-36e0e52b76f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:58:58.455639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8737b9-2b88-4b2c-b648-36e0e52b76f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:58:58.464314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8737b9-2b88-4b2c-b648-36e0e52b76f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:58:58.465772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8737b9-2b88-4b2c-b648-36e0e52b76f1 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:58:58.475980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 733/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:59:00.723251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535da39-9dc8-41b4-9de3-83fcde316d28 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:00.723827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535da39-9dc8-41b4-9de3-83fcde316d28 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:00.723827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535da39-9dc8-41b4-9de3-83fcde316d28 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:00.725256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:59:00.730659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5365e9e-f358-4f0f-859c-8b0bb3774b3f None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:00.731045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5365e9e-f358-4f0f-859c-8b0bb3774b3f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:00.731206 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5365e9e-f358-4f0f-859c-8b0bb3774b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:00.750177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-c5365e9e-f358-4f0f-859c-8b0bb3774b3f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:59:00.751355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c5365e9e-f358-4f0f-859c-8b0bb3774b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:59:00.758918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:59:00.794844 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535da39-9dc8-41b4-9de3-83fcde316d28 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:59:00.805983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535da39-9dc8-41b4-9de3-83fcde316d28 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:59:00.809478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664893a-6d65-41a0-a273-0e0028cf87d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:59:00.810109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664893a-6d65-41a0-a273-0e0028cf87d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:00.810392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664893a-6d65-41a0-a273-0e0028cf87d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:00.810665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664893a-6d65-41a0-a273-0e0028cf87d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:00.820079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664893a-6d65-41a0-a273-0e0028cf87d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:59:00.821546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664893a-6d65-41a0-a273-0e0028cf87d1 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:59:00.830873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 733/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:59:03.134870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16c9d6-a105-4a07-a347-2d926ab019e4 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:03.135434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16c9d6-a105-4a07-a347-2d926ab019e4 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:03.135434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16c9d6-a105-4a07-a347-2d926ab019e4 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:03.136799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 01:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:59:03.142881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0abe2-dcce-4b0e-bee9-6496e55b61e2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:03.143300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0abe2-dcce-4b0e-bee9-6496e55b61e2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:03.143444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0abe2-dcce-4b0e-bee9-6496e55b61e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:03.163582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-83b0abe2-dcce-4b0e-bee9-6496e55b61e2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:59:03.165009 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-83b0abe2-dcce-4b0e-bee9-6496e55b61e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:59:03.173315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 01:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:59:03.211825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16c9d6-a105-4a07-a347-2d926ab019e4 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:59:03.222897 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16c9d6-a105-4a07-a347-2d926ab019e4 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:59:03.226373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b039609-5229-453c-a75e-1f2c3ca2d2f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:59:03.227003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b039609-5229-453c-a75e-1f2c3ca2d2f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:03.227285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b039609-5229-453c-a75e-1f2c3ca2d2f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:03.227558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b039609-5229-453c-a75e-1f2c3ca2d2f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:03.236627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b039609-5229-453c-a75e-1f2c3ca2d2f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:59:03.238055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b039609-5229-453c-a75e-1f2c3ca2d2f4 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:59:03.247160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 736/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:59:13.158444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=11 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:59:13.243179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=14 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 01:59:35.305368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baed14c-8416-4662-ae77-561c5b0f87a3 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:35.305970 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baed14c-8416-4662-ae77-561c5b0f87a3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:35.305970 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baed14c-8416-4662-ae77-561c5b0f87a3 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:35.307273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 735/1471] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:59:35.313280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd450b-fb90-4927-b046-b8317b3de826 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:35.313702 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd450b-fb90-4927-b046-b8317b3de826 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:35.313837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd450b-fb90-4927-b046-b8317b3de826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:35.333379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ddfd450b-fb90-4927-b046-b8317b3de826 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:59:35.334587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ddfd450b-fb90-4927-b046-b8317b3de826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:59:35.342736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 737/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:59:35.390098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c696b-67eb-4500-bd06-c658feb80e81 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:35.390384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c696b-67eb-4500-bd06-c658feb80e81 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:35.390657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c696b-67eb-4500-bd06-c658feb80e81 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:35.392036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 736/1473] 10.222.0.20 () {54 vars in 962 bytes} [Wed Jun 8 01:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:59:35.396775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8a81d-24fd-483d-9648-215bc6a6dd43 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:35.397074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8a81d-24fd-483d-9648-215bc6a6dd43 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:35.397357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8a81d-24fd-483d-9648-215bc6a6dd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:35.416970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-33f8a81d-24fd-483d-9648-215bc6a6dd43 None None] MFA Rules not processed for user `7d1e01f6f82a4ebbb15d0cb55b963af4`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:59:35.418170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-33f8a81d-24fd-483d-9648-215bc6a6dd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:59:35.425240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 738/1474] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Jun 8 01:59:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:59:35.432258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c696b-67eb-4500-bd06-c658feb80e81 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:59:35.443198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c696b-67eb-4500-bd06-c658feb80e81 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:59:35.446619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecdbf7-ec83-467d-a595-fd7afe3eeef8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d1e01f6f82a4ebbb15d0cb55b963af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:59:35.447235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecdbf7-ec83-467d-a595-fd7afe3eeef8 service glance] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:35.447514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecdbf7-ec83-467d-a595-fd7afe3eeef8 service glance] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:35.447784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecdbf7-ec83-467d-a595-fd7afe3eeef8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:35.457083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecdbf7-ec83-467d-a595-fd7afe3eeef8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:59:35.458494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecdbf7-ec83-467d-a595-fd7afe3eeef8 service glance] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:59:35.467571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 737/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:59:37.815978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d1110-d668-4bff-8c3d-533fdad73005 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:37.816572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d1110-d668-4bff-8c3d-533fdad73005 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:37.816572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d1110-d668-4bff-8c3d-533fdad73005 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:37.817930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 739/1476] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 01:59:37.823224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d126e4-e2ba-4a5d-9a23-4df1e07086fa None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:37.823616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d126e4-e2ba-4a5d-9a23-4df1e07086fa None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:37.823783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d126e4-e2ba-4a5d-9a23-4df1e07086fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:37.841627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a6d126e4-e2ba-4a5d-9a23-4df1e07086fa None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:59:37.842827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a6d126e4-e2ba-4a5d-9a23-4df1e07086fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:59:37.850569 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 738/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:59:37.874670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d1110-d668-4bff-8c3d-533fdad73005 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:59:37.887127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d1110-d668-4bff-8c3d-533fdad73005 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:59:37.892201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b592cb-24f1-4b5f-a580-15b22060b9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:59:37.892868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b592cb-24f1-4b5f-a580-15b22060b9ed admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:37.893151 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b592cb-24f1-4b5f-a580-15b22060b9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:37.893432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b592cb-24f1-4b5f-a580-15b22060b9ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:37.894157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b592cb-24f1-4b5f-a580-15b22060b9ed admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:59:37.895882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: WARNING py.warnings [None req-96b592cb-24f1-4b5f-a580-15b22060b9ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 01:59:37.895882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: warnings.warn(msg) Jun 08 01:59:37.895882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: Jun 08 01:59:37.897108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b592cb-24f1-4b5f-a580-15b22060b9ed admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:59:37.916281 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 740/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Jun 8 01:59:37 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 01:59:40.193041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59b47-a487-4d44-8c17-2fb48e3c631e None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:40.193575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59b47-a487-4d44-8c17-2fb48e3c631e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:40.193575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59b47-a487-4d44-8c17-2fb48e3c631e None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:40.195017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 739/1479] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:59:40.201786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17351378-b5c2-43bf-98ca-5f06946763ba None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:40.202551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17351378-b5c2-43bf-98ca-5f06946763ba None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:40.202551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17351378-b5c2-43bf-98ca-5f06946763ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:40.226490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-17351378-b5c2-43bf-98ca-5f06946763ba None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:59:40.227692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-17351378-b5c2-43bf-98ca-5f06946763ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:59:40.236448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 741/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:59:40.280493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a59b47-a487-4d44-8c17-2fb48e3c631e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:59:40.296546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a59b47-a487-4d44-8c17-2fb48e3c631e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:59:40.305581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9302516-631e-498d-aa98-d6c6a5f838f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:59:40.306233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9302516-631e-498d-aa98-d6c6a5f838f3 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:40.306519 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9302516-631e-498d-aa98-d6c6a5f838f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:40.306796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9302516-631e-498d-aa98-d6c6a5f838f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:40.315830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9302516-631e-498d-aa98-d6c6a5f838f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:59:40.317538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9302516-631e-498d-aa98-d6c6a5f838f3 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:59:40.327978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 740/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:59:42.799580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd3bb8-fa00-4f10-9143-d50e4f51d187 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:42.800120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd3bb8-fa00-4f10-9143-d50e4f51d187 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:42.800120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd3bb8-fa00-4f10-9143-d50e4f51d187 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:42.801580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 742/1482] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:59:42.806725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef666f78-ecb4-41cc-bcc0-dcc2a6d8d50d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:42.807169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef666f78-ecb4-41cc-bcc0-dcc2a6d8d50d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:42.807265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef666f78-ecb4-41cc-bcc0-dcc2a6d8d50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:42.826190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ef666f78-ecb4-41cc-bcc0-dcc2a6d8d50d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:59:42.827378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ef666f78-ecb4-41cc-bcc0-dcc2a6d8d50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:59:42.835161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 741/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:59:42.872115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd3bb8-fa00-4f10-9143-d50e4f51d187 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:59:42.883261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd3bb8-fa00-4f10-9143-d50e4f51d187 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:59:42.889323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05727c60-9274-483a-8ab1-07f5c164ebdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:59:42.889993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05727c60-9274-483a-8ab1-07f5c164ebdb service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:42.890315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05727c60-9274-483a-8ab1-07f5c164ebdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:42.890596 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05727c60-9274-483a-8ab1-07f5c164ebdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:42.900356 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05727c60-9274-483a-8ab1-07f5c164ebdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:59:42.901848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05727c60-9274-483a-8ab1-07f5c164ebdb service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:59:42.912286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 743/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 01:59:45.206368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2b6ec-1b93-49e7-b07e-6282e89d04a6 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:45.206879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2b6ec-1b93-49e7-b07e-6282e89d04a6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:45.206879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2b6ec-1b93-49e7-b07e-6282e89d04a6 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:45.208201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 742/1485] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:59:45.213454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c64fe-ea29-4b3e-8089-7b85ded7aa8d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:45.213902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c64fe-ea29-4b3e-8089-7b85ded7aa8d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:45.214011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c64fe-ea29-4b3e-8089-7b85ded7aa8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:45.231446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-cb2c64fe-ea29-4b3e-8089-7b85ded7aa8d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:59:45.232710 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-cb2c64fe-ea29-4b3e-8089-7b85ded7aa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:59:45.240307 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 744/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:59:45.278639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2b6ec-1b93-49e7-b07e-6282e89d04a6 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:59:45.289432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2b6ec-1b93-49e7-b07e-6282e89d04a6 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:59:45.292920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f9fca-1879-480e-b551-db55f62c310b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:59:45.293552 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f9fca-1879-480e-b551-db55f62c310b service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:45.293833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f9fca-1879-480e-b551-db55f62c310b service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:45.294120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f9fca-1879-480e-b551-db55f62c310b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:45.303101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f9fca-1879-480e-b551-db55f62c310b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:59:45.304550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f9fca-1879-480e-b551-db55f62c310b service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:59:45.314681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 743/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:59:47.678588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc50ec2-0f9c-4267-9586-0259f02d9b72 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:47.679245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc50ec2-0f9c-4267-9586-0259f02d9b72 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:47.679422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc50ec2-0f9c-4267-9586-0259f02d9b72 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:47.681133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 745/1488] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:59:47.687360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78545-7089-4c03-bc05-dc00a97a8828 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:47.687818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78545-7089-4c03-bc05-dc00a97a8828 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:47.687926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78545-7089-4c03-bc05-dc00a97a8828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:47.705813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-21f78545-7089-4c03-bc05-dc00a97a8828 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:59:47.706998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-21f78545-7089-4c03-bc05-dc00a97a8828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:59:47.714720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 744/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 01:59:47.754127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc50ec2-0f9c-4267-9586-0259f02d9b72 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:59:47.765230 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc50ec2-0f9c-4267-9586-0259f02d9b72 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:59:47.775163 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103daa2-ddce-45ae-8147-e0f9d56906f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:59:47.775792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103daa2-ddce-45ae-8147-e0f9d56906f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:47.776071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103daa2-ddce-45ae-8147-e0f9d56906f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:47.776342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103daa2-ddce-45ae-8147-e0f9d56906f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:47.785380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103daa2-ddce-45ae-8147-e0f9d56906f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:59:47.786811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103daa2-ddce-45ae-8147-e0f9d56906f5 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:59:47.798492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 746/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:59:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 01:59:50.273884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041151f6-2f95-4d83-a267-4075a073e52a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:50.274428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041151f6-2f95-4d83-a267-4075a073e52a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:50.274428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041151f6-2f95-4d83-a267-4075a073e52a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:50.275819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 745/1491] 10.222.0.20 () {54 vars in 926 bytes} [Wed Jun 8 01:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 01:59:50.282566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b67aaa-466d-4549-b499-836afeb93c66 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:50.283012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b67aaa-466d-4549-b499-836afeb93c66 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:50.283115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b67aaa-466d-4549-b499-836afeb93c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:50.301809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-c2b67aaa-466d-4549-b499-836afeb93c66 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 01:59:50.303008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c2b67aaa-466d-4549-b499-836afeb93c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 01:59:50.310914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 747/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Wed Jun 8 01:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 01:59:50.350412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041151f6-2f95-4d83-a267-4075a073e52a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 01:59:50.360967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041151f6-2f95-4d83-a267-4075a073e52a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 01:59:50.364474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28344bb-f0b4-42b1-aa3f-535e0e081d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 01:59:50.365108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28344bb-f0b4-42b1-aa3f-535e0e081d2e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 01:59:50.365392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28344bb-f0b4-42b1-aa3f-535e0e081d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 01:59:50.365667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28344bb-f0b4-42b1-aa3f-535e0e081d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 01:59:50.374470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28344bb-f0b4-42b1-aa3f-535e0e081d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 01:59:50.375890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28344bb-f0b4-42b1-aa3f-535e0e081d2e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 01:59:50.385308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 746/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 01:59:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:00:00.296668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=33 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:00:00.381034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=33 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:01:44.303436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb02bcc-0342-4d22-bfc6-cd7d814e11cb None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.304021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb02bcc-0342-4d22-bfc6-cd7d814e11cb None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.304021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb02bcc-0342-4d22-bfc6-cd7d814e11cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.322229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-8cb02bcc-0342-4d22-bfc6-cd7d814e11cb None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:01:44.323427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8cb02bcc-0342-4d22-bfc6-cd7d814e11cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:01:44.331754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 748/1494] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:01:44.339817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07a2eb-d650-40de-a9e3-972ebd9e8077 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.340418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07a2eb-d650-40de-a9e3-972ebd9e8077 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.340418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07a2eb-d650-40de-a9e3-972ebd9e8077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.358786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0d07a2eb-d650-40de-a9e3-972ebd9e8077 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:01:44.360002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0d07a2eb-d650-40de-a9e3-972ebd9e8077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:01:44.367816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 747/1495] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:01:44.372928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb02bcc-0342-4d22-bfc6-cd7d814e11cb None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:01:44.382691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb02bcc-0342-4d22-bfc6-cd7d814e11cb None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:01:44.387462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd20df5-7f08-42b2-92c2-44cfaa7744d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:01:44.388110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd20df5-7f08-42b2-92c2-44cfaa7744d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.388411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd20df5-7f08-42b2-92c2-44cfaa7744d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.388702 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd20df5-7f08-42b2-92c2-44cfaa7744d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.389422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd20df5-7f08-42b2-92c2-44cfaa7744d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:01:44.389729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd20df5-7f08-42b2-92c2-44cfaa7744d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:01:44.390756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: WARNING py.warnings [None req-edd20df5-7f08-42b2-92c2-44cfaa7744d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:01:44.390756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: warnings.warn(msg) Jun 08 02:01:44.390756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: Jun 08 02:01:44.391634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd20df5-7f08-42b2-92c2-44cfaa7744d4 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:01:44.403396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 749/1496] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:01:44.411295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07a2eb-d650-40de-a9e3-972ebd9e8077 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:01:44.421309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07a2eb-d650-40de-a9e3-972ebd9e8077 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:01:44.425842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff72556-305d-4dd4-a54a-1874e5f979a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:01:44.426494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72556-305d-4dd4-a54a-1874e5f979a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.426773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72556-305d-4dd4-a54a-1874e5f979a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.427043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72556-305d-4dd4-a54a-1874e5f979a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.427749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff72556-305d-4dd4-a54a-1874e5f979a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:01:44.430507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-aff72556-305d-4dd4-a54a-1874e5f979a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:01:44.430507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 08 02:01:44.430507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: Jun 08 02:01:44.430829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff72556-305d-4dd4-a54a-1874e5f979a9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:01:44.467100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 748/1497] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:01:44 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:01:44.471097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd20df5-7f08-42b2-92c2-44cfaa7744d4 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:01:44.481976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd20df5-7f08-42b2-92c2-44cfaa7744d4 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:01:44.536449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e83a16-73a9-4210-9989-d3243731a2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:01:44.537107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e83a16-73a9-4210-9989-d3243731a2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.537390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e83a16-73a9-4210-9989-d3243731a2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.537665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e83a16-73a9-4210-9989-d3243731a2d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.538608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e83a16-73a9-4210-9989-d3243731a2d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:01:44.539621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: WARNING py.warnings [None req-d5e83a16-73a9-4210-9989-d3243731a2d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:01:44.539621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: warnings.warn(msg) Jun 08 02:01:44.539621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: Jun 08 02:01:44.540842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e83a16-73a9-4210-9989-d3243731a2d5 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:01:44.561301 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 750/1498] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:01:44 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:01:44.565287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff72556-305d-4dd4-a54a-1874e5f979a9 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:01:44.575472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff72556-305d-4dd4-a54a-1874e5f979a9 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:01:44.579966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367565c6-41ff-4bd9-851a-3dc8c374e6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:01:44.580650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367565c6-41ff-4bd9-851a-3dc8c374e6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.580935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367565c6-41ff-4bd9-851a-3dc8c374e6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.581211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367565c6-41ff-4bd9-851a-3dc8c374e6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.581934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367565c6-41ff-4bd9-851a-3dc8c374e6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:01:44.582949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-367565c6-41ff-4bd9-851a-3dc8c374e6b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:01:44.582949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 08 02:01:44.582949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: Jun 08 02:01:44.583831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367565c6-41ff-4bd9-851a-3dc8c374e6b8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:01:44.596864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 749/1499] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:01:44.600722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e83a16-73a9-4210-9989-d3243731a2d5 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:01:44.611043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e83a16-73a9-4210-9989-d3243731a2d5 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:01:44.615550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7f0f1-937b-4da9-86de-7ee7e20c457d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:01:44.616208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7f0f1-937b-4da9-86de-7ee7e20c457d admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.616510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7f0f1-937b-4da9-86de-7ee7e20c457d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.616789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7f0f1-937b-4da9-86de-7ee7e20c457d admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.617504 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7f0f1-937b-4da9-86de-7ee7e20c457d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:01:44.618508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: WARNING py.warnings [None req-4ad7f0f1-937b-4da9-86de-7ee7e20c457d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:01:44.618508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: warnings.warn(msg) Jun 08 02:01:44.618508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: Jun 08 02:01:44.619379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7f0f1-937b-4da9-86de-7ee7e20c457d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:01:44.633039 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 751/1500] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:01:44.637070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367565c6-41ff-4bd9-851a-3dc8c374e6b8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:01:44.646704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367565c6-41ff-4bd9-851a-3dc8c374e6b8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:01:44.651140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951830f-84be-4ddd-ad9b-ef7a36bcb0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:01:44.652033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951830f-84be-4ddd-ad9b-ef7a36bcb0ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.652317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951830f-84be-4ddd-ad9b-ef7a36bcb0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.652698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951830f-84be-4ddd-ad9b-ef7a36bcb0ca admin admin] PATH_INFO: `/v3/projects/70c25109c06c4bcca35802cf55e8bba5/users/5db771c83483426b97d5c6d1b1320de4/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.668113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951830f-84be-4ddd-ad9b-ef7a36bcb0ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c25109c06c4bcca35802cf55e8bba5, user_id=5db771c83483426b97d5c6d1b1320de4, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:01:44.669196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-b951830f-84be-4ddd-ad9b-ef7a36bcb0ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:01:44.669196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 08 02:01:44.669196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: Jun 08 02:01:44.671121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951830f-84be-4ddd-ad9b-ef7a36bcb0ca admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:01:44.684127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 750/1501] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:01:44 2022] PUT /identity/v3/projects/70c25109c06c4bcca35802cf55e8bba5/users/5db771c83483426b97d5c6d1b1320de4/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:01:44.696809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7da6b-079e-4ddd-b448-82d902c18c73 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.697100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7da6b-079e-4ddd-b448-82d902c18c73 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.697376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7da6b-079e-4ddd-b448-82d902c18c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.722736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7fe7da6b-079e-4ddd-b448-82d902c18c73 None None] MFA Rules not processed for user `5db771c83483426b97d5c6d1b1320de4`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:01:44.723866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7fe7da6b-079e-4ddd-b448-82d902c18c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:01:44.787319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 752/1502] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:01:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:01:44.801459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db781b6-3972-4680-938d-b90445a1f408 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.801745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db781b6-3972-4680-938d-b90445a1f408 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.802018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db781b6-3972-4680-938d-b90445a1f408 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.803517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 751/1503] 10.222.0.20 () {54 vars in 963 bytes} [Wed Jun 8 02:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 02:01:44.808191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525c5d5-82ef-4d6c-ae0c-54d68992a6ba None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.808517 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525c5d5-82ef-4d6c-ae0c-54d68992a6ba None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.808803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525c5d5-82ef-4d6c-ae0c-54d68992a6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.826335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3525c5d5-82ef-4d6c-ae0c-54d68992a6ba None None] MFA Rules not processed for user `236226ec865b49febc7713df1c0fab98`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:01:44.827537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3525c5d5-82ef-4d6c-ae0c-54d68992a6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:01:44.870940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:01:44.879338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 753/1504] 10.222.0.20 () {58 vars in 1060 bytes} [Wed Jun 8 02:01:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:01:44.887434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db781b6-3972-4680-938d-b90445a1f408 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:01:44.900614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db781b6-3972-4680-938d-b90445a1f408 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:01:44.905181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689584a-567a-4361-9755-9b6a7262d110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:01:44.905828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689584a-567a-4361-9755-9b6a7262d110 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.906110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689584a-567a-4361-9755-9b6a7262d110 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.906383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689584a-567a-4361-9755-9b6a7262d110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.916098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689584a-567a-4361-9755-9b6a7262d110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:01:44.918170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689584a-567a-4361-9755-9b6a7262d110 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:01:44.928983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 752/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:01:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:01:44.970342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfa30c-e493-4d52-8a23-d2e36c74b882 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:44.970634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfa30c-e493-4d52-8a23-d2e36c74b882 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:44.970916 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfa30c-e493-4d52-8a23-d2e36c74b882 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:44.972184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 754/1506] 10.222.0.20 () {52 vars in 829 bytes} [Wed Jun 8 02:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:01:44.975807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689584a-567a-4361-9755-9b6a7262d110 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:01:45.036080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689584a-567a-4361-9755-9b6a7262d110 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:01:45.040849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:01:45.041705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:45.041986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:45.042255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] PATH_INFO: `/v3/users/5db771c83483426b97d5c6d1b1320de4` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:45.044090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db771c83483426b97d5c6d1b1320de4)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:01:45.045131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:01:45.045131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 08 02:01:45.045131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: Jun 08 02:01:45.046329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:01:45.111017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db771c83483426b97d5c6d1b1320de4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:01:45.111750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db771c83483426b97d5c6d1b1320de4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:01:45.123949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db771c83483426b97d5c6d1b1320de4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:01:45.124618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db771c83483426b97d5c6d1b1320de4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:01:45.133842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db771c83483426b97d5c6d1b1320de4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:01:45.150866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:01:45.161001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 753/1507] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:01:44 2022] DELETE /identity/v3/users/5db771c83483426b97d5c6d1b1320de4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:01:45.165608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f94c57-bc0f-49f8-afc0-0e59042339ec None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:45.165898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f94c57-bc0f-49f8-afc0-0e59042339ec None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:45.166178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f94c57-bc0f-49f8-afc0-0e59042339ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:45.187653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-d5f94c57-bc0f-49f8-afc0-0e59042339ec None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:01:45.188873 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d5f94c57-bc0f-49f8-afc0-0e59042339ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:01:45.246133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 755/1508] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:01:45.256126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:01:45.257612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:01:45.313877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36881651-88b2-4c83-94b9-b848e6db04fc admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:01:45.317456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80f94f-2f83-4b39-b953-16fdc439e1a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:01:45.318108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80f94f-2f83-4b39-b953-16fdc439e1a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:45.318394 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80f94f-2f83-4b39-b953-16fdc439e1a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:45.318674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80f94f-2f83-4b39-b953-16fdc439e1a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:45.328268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80f94f-2f83-4b39-b953-16fdc439e1a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:01:45.329742 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80f94f-2f83-4b39-b953-16fdc439e1a3 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:01:45.339083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 754/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:01:45.667214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f94c57-bc0f-49f8-afc0-0e59042339ec None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:01:45.668912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d5f94c57-bc0f-49f8-afc0-0e59042339ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:01:45.683853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f94c57-bc0f-49f8-afc0-0e59042339ec None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:01:45.689494 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:01:45.690622 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:01:45.690958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:01:45.691237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] PATH_INFO: `/v3/projects/70c25109c06c4bcca35802cf55e8bba5` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:01:45.692459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c25109c06c4bcca35802cf55e8bba5)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:01:45.693490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: WARNING py.warnings [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:01:45.693490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: warnings.warn(msg) Jun 08 02:01:45.693490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: Jun 08 02:01:45.695487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:01:45.755261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c25109c06c4bcca35802cf55e8bba5', 'request_id': 'req-957af856-5661-4708-bc1f-21be1398d263'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:01:45.761968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c25109c06c4bcca35802cf55e8bba5', 'request_id': 'req-957af856-5661-4708-bc1f-21be1398d263'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:01:45.775083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c25109c06c4bcca35802cf55e8bba5', 'request_id': 'req-957af856-5661-4708-bc1f-21be1398d263'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:01:45.791538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] The token cache is being invalidate because project 70c25109c06c4bcca35802cf55e8bba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:01:45.791831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:01:45.792523 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-957af856-5661-4708-bc1f-21be1398d263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:01:45.794246 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 756/1510] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:01:45 2022] DELETE /identity/v3/projects/70c25109c06c4bcca35802cf55e8bba5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:01:55.334924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:01:55.787934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:03:18.850787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b3fe-a727-4d40-b797-e2b7183890ea None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:18.851380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b3fe-a727-4d40-b797-e2b7183890ea None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:18.851380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b3fe-a727-4d40-b797-e2b7183890ea None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:18.853763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 02:03:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:03:18.858532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9855344-7cab-48c9-93ce-01a8a229ca48 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:18.858941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9855344-7cab-48c9-93ce-01a8a229ca48 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:18.859099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9855344-7cab-48c9-93ce-01a8a229ca48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:18.877771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b9855344-7cab-48c9-93ce-01a8a229ca48 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:03:18.878976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b9855344-7cab-48c9-93ce-01a8a229ca48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:18.946889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 02:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:03:19.116196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931b3fe-a727-4d40-b797-e2b7183890ea None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:03:19.117891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4931b3fe-a727-4d40-b797-e2b7183890ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:19.191920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931b3fe-a727-4d40-b797-e2b7183890ea None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:03:19.197485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cef9c5-3c30-4979-b805-531c042bedb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:03:19.198144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cef9c5-3c30-4979-b805-531c042bedb9 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:19.198433 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cef9c5-3c30-4979-b805-531c042bedb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:19.198711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cef9c5-3c30-4979-b805-531c042bedb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:19.206892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cef9c5-3c30-4979-b805-531c042bedb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:03:19.208541 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cef9c5-3c30-4979-b805-531c042bedb9 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:03:19.218963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 756/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:03:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:03:20.105210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf9f6-f034-427e-813d-715e0b50b986 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:20.105769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf9f6-f034-427e-813d-715e0b50b986 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:20.105769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf9f6-f034-427e-813d-715e0b50b986 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:20.107132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 02:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:03:20.112046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c6ce2-ca3e-4d96-bc60-5eadc804ec89 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:20.112516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c6ce2-ca3e-4d96-bc60-5eadc804ec89 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:20.112674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c6ce2-ca3e-4d96-bc60-5eadc804ec89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:20.131796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b72c6ce2-ca3e-4d96-bc60-5eadc804ec89 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:03:20.133118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b72c6ce2-ca3e-4d96-bc60-5eadc804ec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:20.143740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 02:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:03:20.295832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaf9f6-f034-427e-813d-715e0b50b986 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:03:20.311293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaf9f6-f034-427e-813d-715e0b50b986 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:03:20.316720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5092d-ffb2-4d6f-b3dc-4e1b10ea8227 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:03:20.317359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5092d-ffb2-4d6f-b3dc-4e1b10ea8227 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:20.317647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5092d-ffb2-4d6f-b3dc-4e1b10ea8227 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:20.317927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5092d-ffb2-4d6f-b3dc-4e1b10ea8227 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:20.328562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5092d-ffb2-4d6f-b3dc-4e1b10ea8227 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:03:20.330556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5092d-ffb2-4d6f-b3dc-4e1b10ea8227 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:03:20.341775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 759/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:03:20.417827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4a8f3-6b70-4a4d-a8a9-f1d86743f23f None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:20.418121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4a8f3-6b70-4a4d-a8a9-f1d86743f23f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:20.418404 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4a8f3-6b70-4a4d-a8a9-f1d86743f23f None None] PATH_INFO: `/v3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:20.419731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 758/1517] 10.222.0.15 () {54 vars in 925 bytes} [Wed Jun 8 02:03:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:03:20.433749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1e741-117a-425f-9b5a-dc6c460d1385 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:20.434034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1e741-117a-425f-9b5a-dc6c460d1385 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:20.434311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1e741-117a-425f-9b5a-dc6c460d1385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:20.452918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-92b1e741-117a-425f-9b5a-dc6c460d1385 None None] MFA Rules not processed for user `4eb69af145154c6d85bd4b845979e6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:03:20.454117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-92b1e741-117a-425f-9b5a-dc6c460d1385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:20.514690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 760/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Jun 8 02:03:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:03:24.638860 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbf648-46f3-440f-a513-b9ade78005d8 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:24.639459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbf648-46f3-440f-a513-b9ade78005d8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:24.639459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbf648-46f3-440f-a513-b9ade78005d8 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:24.640849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 759/1519] 10.222.0.20 () {54 vars in 960 bytes} [Wed Jun 8 02:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:03:24.645859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432ee8-2c15-48c3-9a1a-f4d4141d04b0 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:24.646284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432ee8-2c15-48c3-9a1a-f4d4141d04b0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:24.646415 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432ee8-2c15-48c3-9a1a-f4d4141d04b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:24.665935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7e432ee8-2c15-48c3-9a1a-f4d4141d04b0 None None] MFA Rules not processed for user `4eb69af145154c6d85bd4b845979e6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:03:24.667136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7e432ee8-2c15-48c3-9a1a-f4d4141d04b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:24.675673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 761/1520] 10.222.0.20 () {58 vars in 1057 bytes} [Wed Jun 8 02:03:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:03:24.683114 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbf648-46f3-440f-a513-b9ade78005d8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:03:24.696276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbf648-46f3-440f-a513-b9ade78005d8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:03:24.701473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ad3df-87e2-493e-9946-f9369eda6bad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:03:24.702107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad3df-87e2-493e-9946-f9369eda6bad service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:24.702391 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad3df-87e2-493e-9946-f9369eda6bad service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:24.702667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ad3df-87e2-493e-9946-f9369eda6bad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:24.710944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ad3df-87e2-493e-9946-f9369eda6bad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:03:24.713158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ad3df-87e2-493e-9946-f9369eda6bad service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:03:24.722692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 760/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:03:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:03:31.617924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a0857-33d7-4e0a-8ace-57596dee64e5 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:31.619104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a0857-33d7-4e0a-8ace-57596dee64e5 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:31.619104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a0857-33d7-4e0a-8ace-57596dee64e5 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:31.620372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 02:03:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:03:31.625118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0c7d8-6120-4af2-a23a-2be68a7fe639 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:31.625558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0c7d8-6120-4af2-a23a-2be68a7fe639 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:31.625710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0c7d8-6120-4af2-a23a-2be68a7fe639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:31.643381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2af0c7d8-6120-4af2-a23a-2be68a7fe639 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:03:31.644596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2af0c7d8-6120-4af2-a23a-2be68a7fe639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:31.652384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 02:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:03:31.803471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a0857-33d7-4e0a-8ace-57596dee64e5 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:03:31.818682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a0857-33d7-4e0a-8ace-57596dee64e5 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:03:31.824020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6121e22-74ce-4d9e-a289-013ddc4d66b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:03:31.824682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6121e22-74ce-4d9e-a289-013ddc4d66b2 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:31.824964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6121e22-74ce-4d9e-a289-013ddc4d66b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:31.825240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6121e22-74ce-4d9e-a289-013ddc4d66b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:31.833121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6121e22-74ce-4d9e-a289-013ddc4d66b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:03:31.834699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6121e22-74ce-4d9e-a289-013ddc4d66b2 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:03:31.844164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 763/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:03:32.718984 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3d992-8c14-4627-b05a-c37262c87045 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:32.719512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3d992-8c14-4627-b05a-c37262c87045 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:32.719512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3d992-8c14-4627-b05a-c37262c87045 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:32.720882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 8 02:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:03:32.725506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e04b2-6736-4c79-911a-16c0c51700b3 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:32.725940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e04b2-6736-4c79-911a-16c0c51700b3 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:32.726045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e04b2-6736-4c79-911a-16c0c51700b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:32.744683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-a90e04b2-6736-4c79-911a-16c0c51700b3 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:03:32.745865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a90e04b2-6736-4c79-911a-16c0c51700b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:32.753539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 8 02:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:03:32.905467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3d992-8c14-4627-b05a-c37262c87045 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:03:32.921785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3d992-8c14-4627-b05a-c37262c87045 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:03:32.928126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53530346-1fb8-4f51-9b44-cde45fe36ea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:03:32.928820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53530346-1fb8-4f51-9b44-cde45fe36ea2 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:32.929107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53530346-1fb8-4f51-9b44-cde45fe36ea2 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:32.929381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53530346-1fb8-4f51-9b44-cde45fe36ea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:32.938267 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53530346-1fb8-4f51-9b44-cde45fe36ea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:03:32.939838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53530346-1fb8-4f51-9b44-cde45fe36ea2 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:03:32.949518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 763/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:03:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:03:36.854854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12abd29-6454-4f5b-a721-5e3d43db7677 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:36.855384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12abd29-6454-4f5b-a721-5e3d43db7677 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:36.855494 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12abd29-6454-4f5b-a721-5e3d43db7677 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:36.856831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 02:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:03:36.862097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c46e69-2a82-40a3-a950-c5697e9963a7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:36.862536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c46e69-2a82-40a3-a950-c5697e9963a7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:36.862663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c46e69-2a82-40a3-a950-c5697e9963a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:36.881315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f9c46e69-2a82-40a3-a950-c5697e9963a7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:03:36.882548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f9c46e69-2a82-40a3-a950-c5697e9963a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:36.891389 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 02:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:03:36.929160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12abd29-6454-4f5b-a721-5e3d43db7677 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:03:36.930988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e12abd29-6454-4f5b-a721-5e3d43db7677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:36.988949 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12abd29-6454-4f5b-a721-5e3d43db7677 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:03:36.992489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a37ff-5aa1-419d-b36c-d69067eb5fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:03:36.993445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a37ff-5aa1-419d-b36c-d69067eb5fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:36.993732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a37ff-5aa1-419d-b36c-d69067eb5fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:36.994007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a37ff-5aa1-419d-b36c-d69067eb5fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:37.002808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a37ff-5aa1-419d-b36c-d69067eb5fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:03:37.004229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a37ff-5aa1-419d-b36c-d69067eb5fb2 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:03:37.009951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:03:37.015223 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 766/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:03:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:03:39.247488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda0886-3217-42ca-b3c9-f0c7284f4a1a None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:39.248019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda0886-3217-42ca-b3c9-f0c7284f4a1a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:39.248128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda0886-3217-42ca-b3c9-f0c7284f4a1a None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:39.249465 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 02:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:03:39.254739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588483e-a521-493e-ab75-bdc07b2c8af7 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:39.255143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588483e-a521-493e-ab75-bdc07b2c8af7 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:39.255289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588483e-a521-493e-ab75-bdc07b2c8af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:39.274553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-8588483e-a521-493e-ab75-bdc07b2c8af7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:03:39.275755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8588483e-a521-493e-ab75-bdc07b2c8af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:39.283599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 02:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:03:39.319711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda0886-3217-42ca-b3c9-f0c7284f4a1a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:03:39.322136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7cda0886-3217-42ca-b3c9-f0c7284f4a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:39.335012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda0886-3217-42ca-b3c9-f0c7284f4a1a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:03:39.339252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4f20c-90db-4343-8a5d-af0a5c39e371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:03:39.339896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4f20c-90db-4343-8a5d-af0a5c39e371 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:39.340184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4f20c-90db-4343-8a5d-af0a5c39e371 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:39.340501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4f20c-90db-4343-8a5d-af0a5c39e371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:39.349496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4f20c-90db-4343-8a5d-af0a5c39e371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:03:39.350937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4f20c-90db-4343-8a5d-af0a5c39e371 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:03:39.360307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 766/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:03:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:03:43.051000 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cc5f7-2afb-4d0d-a8f9-193c9a5026ce None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:43.051531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cc5f7-2afb-4d0d-a8f9-193c9a5026ce None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:43.051531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cc5f7-2afb-4d0d-a8f9-193c9a5026ce None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:43.052936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 02:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:03:43.058668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3f851-4ced-4f42-afd0-222a650c9f8b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:43.059090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3f851-4ced-4f42-afd0-222a650c9f8b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:43.059229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3f851-4ced-4f42-afd0-222a650c9f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:43.077769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-96d3f851-4ced-4f42-afd0-222a650c9f8b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:03:43.078945 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-96d3f851-4ced-4f42-afd0-222a650c9f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:43.086521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 02:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:03:43.359285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cc5f7-2afb-4d0d-a8f9-193c9a5026ce None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:03:43.361096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-521cc5f7-2afb-4d0d-a8f9-193c9a5026ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:43.420619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cc5f7-2afb-4d0d-a8f9-193c9a5026ce None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:03:43.424097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d9600-54b2-4c3d-8fdc-b0240bcf7071 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:03:43.424767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d9600-54b2-4c3d-8fdc-b0240bcf7071 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:43.425050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d9600-54b2-4c3d-8fdc-b0240bcf7071 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:43.425323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d9600-54b2-4c3d-8fdc-b0240bcf7071 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:43.434928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d9600-54b2-4c3d-8fdc-b0240bcf7071 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:03:43.436353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d9600-54b2-4c3d-8fdc-b0240bcf7071 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:03:43.445645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 769/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:03:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:03:45.704425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b1ef1-4b07-4e12-8655-dd0b53951a63 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:45.704960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b1ef1-4b07-4e12-8655-dd0b53951a63 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:45.705060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b1ef1-4b07-4e12-8655-dd0b53951a63 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:45.706437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 8 02:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:03:45.711853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0d206-6155-43b1-bcb0-af9b5367f078 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:45.712243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0d206-6155-43b1-bcb0-af9b5367f078 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:45.712437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0d206-6155-43b1-bcb0-af9b5367f078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:45.731492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-4ad0d206-6155-43b1-bcb0-af9b5367f078 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:03:45.732743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4ad0d206-6155-43b1-bcb0-af9b5367f078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:45.740635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 8 02:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:03:46.009017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b1ef1-4b07-4e12-8655-dd0b53951a63 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:03:46.010979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b38b1ef1-4b07-4e12-8655-dd0b53951a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:03:46.019172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=65 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:03:46.025774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b1ef1-4b07-4e12-8655-dd0b53951a63 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:03:46.029347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784dfb9-5498-433b-8046-c7b46d198633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:03:46.029989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784dfb9-5498-433b-8046-c7b46d198633 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:03:46.030273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784dfb9-5498-433b-8046-c7b46d198633 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:03:46.030550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784dfb9-5498-433b-8046-c7b46d198633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:03:46.039711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784dfb9-5498-433b-8046-c7b46d198633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:03:46.041703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784dfb9-5498-433b-8046-c7b46d198633 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:03:46.051084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 769/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:03:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:03:55.726029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=32 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:03:56.046917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:18.276874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832a9b2-cf01-42e8-badc-16c47d5b6c5b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.278223 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832a9b2-cf01-42e8-badc-16c47d5b6c5b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.278738 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832a9b2-cf01-42e8-badc-16c47d5b6c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.309684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-8832a9b2-cf01-42e8-badc-16c47d5b6c5b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:18.311466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8832a9b2-cf01-42e8-badc-16c47d5b6c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:18.321173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 771/1540] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:18.330489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b93a89-e8c6-4b32-b4d3-925c14577244 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.331072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b93a89-e8c6-4b32-b4d3-925c14577244 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.331072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b93a89-e8c6-4b32-b4d3-925c14577244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.368753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d1b93a89-e8c6-4b32-b4d3-925c14577244 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:18.370627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d1b93a89-e8c6-4b32-b4d3-925c14577244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:18.371055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa4204-5988-4e16-b3da-f1a49935aa34 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.371354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa4204-5988-4e16-b3da-f1a49935aa34 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.371640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa4204-5988-4e16-b3da-f1a49935aa34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.390338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 770/1541] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:18.396247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b93a89-e8c6-4b32-b4d3-925c14577244 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:18.411997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-27fa4204-5988-4e16-b3da-f1a49935aa34 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:18.413601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b93a89-e8c6-4b32-b4d3-925c14577244 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:18.414145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-27fa4204-5988-4e16-b3da-f1a49935aa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:18.421866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8569e-8a4e-456a-9cd9-689aace4d95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:18.423209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8569e-8a4e-456a-9cd9-689aace4d95a admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.423673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8569e-8a4e-456a-9cd9-689aace4d95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.424123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8569e-8a4e-456a-9cd9-689aace4d95a admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.425091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8569e-8a4e-456a-9cd9-689aace4d95a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:18.425581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8569e-8a4e-456a-9cd9-689aace4d95a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:18.426962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-00f8569e-8a4e-456a-9cd9-689aace4d95a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:05:18.426962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 08 02:05:18.426962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: Jun 08 02:05:18.428430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8569e-8a4e-456a-9cd9-689aace4d95a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:18.439993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 772/1542] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:18.447428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22e378-5be4-4253-a752-89ea198ed528 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.447859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22e378-5be4-4253-a752-89ea198ed528 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.448244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22e378-5be4-4253-a752-89ea198ed528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.449674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 771/1543] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:18.459478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8569e-8a4e-456a-9cd9-689aace4d95a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:18.472759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8569e-8a4e-456a-9cd9-689aace4d95a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:18.485721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7a22e378-5be4-4253-a752-89ea198ed528 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:18.487105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7a22e378-5be4-4253-a752-89ea198ed528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:18.487449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd23de-f609-4a5a-b784-cda1b473d9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:18.488297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd23de-f609-4a5a-b784-cda1b473d9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.488759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd23de-f609-4a5a-b784-cda1b473d9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.489200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd23de-f609-4a5a-b784-cda1b473d9a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.490092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd23de-f609-4a5a-b784-cda1b473d9a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:18.492220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd23de-f609-4a5a-b784-cda1b473d9a4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:18.507677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 773/1544] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:18.511982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22e378-5be4-4253-a752-89ea198ed528 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:18.527227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22e378-5be4-4253-a752-89ea198ed528 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:18.540203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f1419-4a8e-40fc-919c-3ab2864e5d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:18.540203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f1419-4a8e-40fc-919c-3ab2864e5d71 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.540203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f1419-4a8e-40fc-919c-3ab2864e5d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.540203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f1419-4a8e-40fc-919c-3ab2864e5d71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.540748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f1419-4a8e-40fc-919c-3ab2864e5d71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:18.540748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f1419-4a8e-40fc-919c-3ab2864e5d71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:18.541923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f1419-4a8e-40fc-919c-3ab2864e5d71 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:18.543349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 772/1545] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:05:18.548013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd23de-f609-4a5a-b784-cda1b473d9a4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:18.565256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 774/1546] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:18.573761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f1419-4a8e-40fc-919c-3ab2864e5d71 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:18.596695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f1419-4a8e-40fc-919c-3ab2864e5d71 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:18.649896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd23de-f609-4a5a-b784-cda1b473d9a4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:18.666667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75946a28-f01a-462c-8393-ab26df3f7bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:18.667759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75946a28-f01a-462c-8393-ab26df3f7bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.668287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75946a28-f01a-462c-8393-ab26df3f7bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.668801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75946a28-f01a-462c-8393-ab26df3f7bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.669724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75946a28-f01a-462c-8393-ab26df3f7bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:18.670983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-75946a28-f01a-462c-8393-ab26df3f7bb0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:05:18.670983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 08 02:05:18.670983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: Jun 08 02:05:18.672643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75946a28-f01a-462c-8393-ab26df3f7bb0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:18.693740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45c7b9-6da2-4d3e-a8c4-fce281bb783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:18.694535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45c7b9-6da2-4d3e-a8c4-fce281bb783c admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.694983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45c7b9-6da2-4d3e-a8c4-fce281bb783c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.695284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45c7b9-6da2-4d3e-a8c4-fce281bb783c admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.696041 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45c7b9-6da2-4d3e-a8c4-fce281bb783c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:18.697094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: WARNING py.warnings [None req-dd45c7b9-6da2-4d3e-a8c4-fce281bb783c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:05:18.697094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: warnings.warn(msg) Jun 08 02:05:18.697094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: Jun 08 02:05:18.698377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45c7b9-6da2-4d3e-a8c4-fce281bb783c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:18.713243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 773/1547] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:05:18.717046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45864d8-1491-4341-bdbd-0e2b218c6bf8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.717597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45864d8-1491-4341-bdbd-0e2b218c6bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.718150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45864d8-1491-4341-bdbd-0e2b218c6bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.751112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a45864d8-1491-4341-bdbd-0e2b218c6bf8 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:18.752321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a45864d8-1491-4341-bdbd-0e2b218c6bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:18.754343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 775/1548] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:18.755237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45c7b9-6da2-4d3e-a8c4-fce281bb783c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:18.830240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 774/1549] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:18.831821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb399e1d-b1b6-4c69-b226-9b717e470c7f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.832117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb399e1d-b1b6-4c69-b226-9b717e470c7f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.832407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb399e1d-b1b6-4c69-b226-9b717e470c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.838046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45c7b9-6da2-4d3e-a8c4-fce281bb783c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:18.843003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407c825-bf69-4a9b-b8e8-7135d34cee6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:18.843763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407c825-bf69-4a9b-b8e8-7135d34cee6f admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.844113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407c825-bf69-4a9b-b8e8-7135d34cee6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.844490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407c825-bf69-4a9b-b8e8-7135d34cee6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.845295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407c825-bf69-4a9b-b8e8-7135d34cee6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:18.846992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407c825-bf69-4a9b-b8e8-7135d34cee6f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:18.855072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-eb399e1d-b1b6-4c69-b226-9b717e470c7f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:18.855072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-eb399e1d-b1b6-4c69-b226-9b717e470c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:18.861059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 776/1550] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:18.862588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1aaef-c7bc-440e-a6a3-9006a2b99d59 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.862877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1aaef-c7bc-440e-a6a3-9006a2b99d59 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.863154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1aaef-c7bc-440e-a6a3-9006a2b99d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.865466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 775/1551] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:18.867696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7943fd4-40d1-4403-b8cf-2328098ec445 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.867991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7943fd4-40d1-4403-b8cf-2328098ec445 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.868272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7943fd4-40d1-4403-b8cf-2328098ec445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.883681 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-10b1aaef-c7bc-440e-a6a3-9006a2b99d59 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:18.884923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-10b1aaef-c7bc-440e-a6a3-9006a2b99d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:18.888965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f7943fd4-40d1-4403-b8cf-2328098ec445 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:18.890146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f7943fd4-40d1-4403-b8cf-2328098ec445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:18.893847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 777/1552] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:18.894609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1aaef-c7bc-440e-a6a3-9006a2b99d59 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:18.903893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 776/1553] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:18.904866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1aaef-c7bc-440e-a6a3-9006a2b99d59 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:18.905243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f5719-2a0f-454c-b61d-911e62219370 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.905539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f5719-2a0f-454c-b61d-911e62219370 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.905848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f5719-2a0f-454c-b61d-911e62219370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.909496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7a069-aa21-4d98-af1b-5d2bd31dc6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:18.910155 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7a069-aa21-4d98-af1b-5d2bd31dc6fe admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.910440 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7a069-aa21-4d98-af1b-5d2bd31dc6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.910724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7a069-aa21-4d98-af1b-5d2bd31dc6fe admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.911671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7a069-aa21-4d98-af1b-5d2bd31dc6fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:18.913687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7a069-aa21-4d98-af1b-5d2bd31dc6fe admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:18.929524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-462f5719-2a0f-454c-b61d-911e62219370 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:18.930763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-462f5719-2a0f-454c-b61d-911e62219370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:18.941927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 777/1554] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:18.945700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 778/1555] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:18.945868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e2689-3225-4d85-89e7-8a1b2a571d2c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.945969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7a069-aa21-4d98-af1b-5d2bd31dc6fe admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:18.946063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e2689-3225-4d85-89e7-8a1b2a571d2c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.946063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e2689-3225-4d85-89e7-8a1b2a571d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.958088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7a069-aa21-4d98-af1b-5d2bd31dc6fe admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:18.964165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e6835-44ba-466e-91ff-28754220d3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:18.964994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e6835-44ba-466e-91ff-28754220d3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.965281 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e6835-44ba-466e-91ff-28754220d3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.965559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e6835-44ba-466e-91ff-28754220d3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.966291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e6835-44ba-466e-91ff-28754220d3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:18.967914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e6835-44ba-466e-91ff-28754220d3a8 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:18.968475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0e9e2689-3225-4d85-89e7-8a1b2a571d2c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:18.969732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0e9e2689-3225-4d85-89e7-8a1b2a571d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:18.978470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 778/1556] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:18.982146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff72d7-4fc5-423c-8079-675c4ba98380 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.982146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff72d7-4fc5-423c-8079-675c4ba98380 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.982146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff72d7-4fc5-423c-8079-675c4ba98380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.982435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 779/1557] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:18.982435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa8080-47a5-489a-b0ed-3dcf2531397c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:18.982435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa8080-47a5-489a-b0ed-3dcf2531397c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:18.983025 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa8080-47a5-489a-b0ed-3dcf2531397c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:18.998838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-51ff72d7-4fc5-423c-8079-675c4ba98380 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:19.000040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-51ff72d7-4fc5-423c-8079-675c4ba98380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:19.000729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b4fa8080-47a5-489a-b0ed-3dcf2531397c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:19.001884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b4fa8080-47a5-489a-b0ed-3dcf2531397c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:19.008689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 779/1558] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:19.009958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099312e-0822-4ad1-9d54-8c4d17982b4a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.013744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099312e-0822-4ad1-9d54-8c4d17982b4a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.013884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 780/1559] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:19.013990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099312e-0822-4ad1-9d54-8c4d17982b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.014078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6cc53-76a1-4a6b-a53d-af7555763672 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.014078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6cc53-76a1-4a6b-a53d-af7555763672 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.014078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6cc53-76a1-4a6b-a53d-af7555763672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.026033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:19.032000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6099312e-0822-4ad1-9d54-8c4d17982b4a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:19.032792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-35b6cc53-76a1-4a6b-a53d-af7555763672 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:19.033254 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6099312e-0822-4ad1-9d54-8c4d17982b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:19.034608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-35b6cc53-76a1-4a6b-a53d-af7555763672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:19.041243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 780/1560] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:19.042009 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099312e-0822-4ad1-9d54-8c4d17982b4a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.046255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 781/1561] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:19.046255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bcdbb-37d7-4eb7-bbed-058bbb4a925b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.046255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bcdbb-37d7-4eb7-bbed-058bbb4a925b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.046255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bcdbb-37d7-4eb7-bbed-058bbb4a925b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.051516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099312e-0822-4ad1-9d54-8c4d17982b4a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.056161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a27353-4581-46fc-8473-dd19da78918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.056851 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a27353-4581-46fc-8473-dd19da78918e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.057132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a27353-4581-46fc-8473-dd19da78918e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.057751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a27353-4581-46fc-8473-dd19da78918e admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.058116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a27353-4581-46fc-8473-dd19da78918e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.059734 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a27353-4581-46fc-8473-dd19da78918e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.061860 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:19.063245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-816bcdbb-37d7-4eb7-bbed-058bbb4a925b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:19.064992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-816bcdbb-37d7-4eb7-bbed-058bbb4a925b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:19.071725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 781/1562] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:19.072897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a27353-4581-46fc-8473-dd19da78918e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.075315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 782/1563] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:19.076102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bcdbb-37d7-4eb7-bbed-058bbb4a925b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.083789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a27353-4581-46fc-8473-dd19da78918e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.086322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bcdbb-37d7-4eb7-bbed-058bbb4a925b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.089093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7500f8-9edd-4554-a7fe-22b3fa4d8728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.089958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7500f8-9edd-4554-a7fe-22b3fa4d8728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.090258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7500f8-9edd-4554-a7fe-22b3fa4d8728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.090533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7500f8-9edd-4554-a7fe-22b3fa4d8728 admin admin] PATH_INFO: `/v3/projects/c1f3f7571cb24d069d7bb1182bafb300/users/fdf621d5692b4fe9995b026819e71f92/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.090976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd9e93-5399-45fc-986c-d155ae735dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.091610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd9e93-5399-45fc-986c-d155ae735dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.091894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd9e93-5399-45fc-986c-d155ae735dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.092169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd9e93-5399-45fc-986c-d155ae735dd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.092923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd9e93-5399-45fc-986c-d155ae735dd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:19.093231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd9e93-5399-45fc-986c-d155ae735dd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.094830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd9e93-5399-45fc-986c-d155ae735dd1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.104816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7500f8-9edd-4554-a7fe-22b3fa4d8728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f3f7571cb24d069d7bb1182bafb300, user_id=fdf621d5692b4fe9995b026819e71f92, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.106908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 783/1564] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:19.107442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7500f8-9edd-4554-a7fe-22b3fa4d8728 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.107715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd9e93-5399-45fc-986c-d155ae735dd1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.118365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd9e93-5399-45fc-986c-d155ae735dd1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.123127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 782/1565] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:19 2022] PUT /identity/v3/projects/c1f3f7571cb24d069d7bb1182bafb300/users/fdf621d5692b4fe9995b026819e71f92/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:19.124390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b34526-cab4-46b5-8438-5c8c52b962a5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.124710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b34526-cab4-46b5-8438-5c8c52b962a5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.124972 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b34526-cab4-46b5-8438-5c8c52b962a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.148112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-93b34526-cab4-46b5-8438-5c8c52b962a5 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:19.149333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-93b34526-cab4-46b5-8438-5c8c52b962a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:19.178400 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a17ae2-639e-4839-b35a-2b4c13b5d54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.179091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a17ae2-639e-4839-b35a-2b4c13b5d54b admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.179407 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a17ae2-639e-4839-b35a-2b4c13b5d54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.179685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a17ae2-639e-4839-b35a-2b4c13b5d54b admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.180430 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a17ae2-639e-4839-b35a-2b4c13b5d54b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:19.180742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a17ae2-639e-4839-b35a-2b4c13b5d54b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.182335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a17ae2-639e-4839-b35a-2b4c13b5d54b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.193745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 784/1566] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:19.194708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a17ae2-639e-4839-b35a-2b4c13b5d54b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.206009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a17ae2-639e-4839-b35a-2b4c13b5d54b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.209122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 783/1567] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:19.210582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b34526-cab4-46b5-8438-5c8c52b962a5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.212120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15adba6-7ac4-4e1e-aed0-2cc81381954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.212893 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15adba6-7ac4-4e1e-aed0-2cc81381954d admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.213245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15adba6-7ac4-4e1e-aed0-2cc81381954d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.213730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15adba6-7ac4-4e1e-aed0-2cc81381954d admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.214449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15adba6-7ac4-4e1e-aed0-2cc81381954d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:19.215021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15adba6-7ac4-4e1e-aed0-2cc81381954d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.216504 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15adba6-7ac4-4e1e-aed0-2cc81381954d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.219652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b34526-cab4-46b5-8438-5c8c52b962a5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.224142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317650a-d47a-4f90-8bcb-d1a0a44e9b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.224824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317650a-d47a-4f90-8bcb-d1a0a44e9b48 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.225107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317650a-d47a-4f90-8bcb-d1a0a44e9b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.226003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317650a-d47a-4f90-8bcb-d1a0a44e9b48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.226720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317650a-d47a-4f90-8bcb-d1a0a44e9b48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:19.227031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317650a-d47a-4f90-8bcb-d1a0a44e9b48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.228644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317650a-d47a-4f90-8bcb-d1a0a44e9b48 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.237174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 785/1568] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:19.237174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15adba6-7ac4-4e1e-aed0-2cc81381954d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.240952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15adba6-7ac4-4e1e-aed0-2cc81381954d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.246368 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd26ea-a5ec-4a88-a31b-0609d4ae256d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.247033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd26ea-a5ec-4a88-a31b-0609d4ae256d admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.247321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd26ea-a5ec-4a88-a31b-0609d4ae256d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.247602 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd26ea-a5ec-4a88-a31b-0609d4ae256d admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.252668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 784/1569] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:19.252809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd26ea-a5ec-4a88-a31b-0609d4ae256d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.252925 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317650a-d47a-4f90-8bcb-d1a0a44e9b48 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.253017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd26ea-a5ec-4a88-a31b-0609d4ae256d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.261577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 786/1570] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:19.262407 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd26ea-a5ec-4a88-a31b-0609d4ae256d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.263200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317650a-d47a-4f90-8bcb-d1a0a44e9b48 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.268166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70de10-c2de-4b7d-9c3a-76e66928d870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.269020 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70de10-c2de-4b7d-9c3a-76e66928d870 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.269458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70de10-c2de-4b7d-9c3a-76e66928d870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.269907 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70de10-c2de-4b7d-9c3a-76e66928d870 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.270635 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70de10-c2de-4b7d-9c3a-76e66928d870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:19.270943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70de10-c2de-4b7d-9c3a-76e66928d870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.273098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70de10-c2de-4b7d-9c3a-76e66928d870 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.273598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd26ea-a5ec-4a88-a31b-0609d4ae256d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.278664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3f63f-88bb-4719-8852-5a15765cdef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.279314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3f63f-88bb-4719-8852-5a15765cdef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.279599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3f63f-88bb-4719-8852-5a15765cdef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.279877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3f63f-88bb-4719-8852-5a15765cdef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.280602 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3f63f-88bb-4719-8852-5a15765cdef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.282558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3f63f-88bb-4719-8852-5a15765cdef5 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.293152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 785/1571] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:19.293152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d47d8a-a2ca-4b71-a79c-149020b3638d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.293152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d47d8a-a2ca-4b71-a79c-149020b3638d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.293152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d47d8a-a2ca-4b71-a79c-149020b3638d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.313177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 787/1572] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.313177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3f63f-88bb-4719-8852-5a15765cdef5 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.316716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e8d47d8a-a2ca-4b71-a79c-149020b3638d None None] MFA Rules not processed for user `fdf621d5692b4fe9995b026819e71f92`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:19.317767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e8d47d8a-a2ca-4b71-a79c-149020b3638d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:19.374414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3f63f-88bb-4719-8852-5a15765cdef5 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.379133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dc3ae-175b-45ed-8fb4-9ca30a885b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.379959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc3ae-175b-45ed-8fb4-9ca30a885b49 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.380254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc3ae-175b-45ed-8fb4-9ca30a885b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.380579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc3ae-175b-45ed-8fb4-9ca30a885b49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.381275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49dc3ae-175b-45ed-8fb4-9ca30a885b49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.383974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49dc3ae-175b-45ed-8fb4-9ca30a885b49 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.384607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 786/1573] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:19.385459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d47d8a-a2ca-4b71-a79c-149020b3638d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.395650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d47d8a-a2ca-4b71-a79c-149020b3638d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.400451 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b88192-daa5-4c77-a7db-8eaa6c89d234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.401111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b88192-daa5-4c77-a7db-8eaa6c89d234 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.401396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b88192-daa5-4c77-a7db-8eaa6c89d234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.401672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b88192-daa5-4c77-a7db-8eaa6c89d234 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.402390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b88192-daa5-4c77-a7db-8eaa6c89d234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:19.402697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b88192-daa5-4c77-a7db-8eaa6c89d234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.404311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b88192-daa5-4c77-a7db-8eaa6c89d234 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.417675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 787/1574] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:19.418640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b88192-daa5-4c77-a7db-8eaa6c89d234 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.419148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 788/1575] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.420513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dc3ae-175b-45ed-8fb4-9ca30a885b49 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.432348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b88192-daa5-4c77-a7db-8eaa6c89d234 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.436863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:19.455525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:19.489244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dc3ae-175b-45ed-8fb4-9ca30a885b49 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.494471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fbb1d-f164-4a05-984d-2a052889e919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.495181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fbb1d-f164-4a05-984d-2a052889e919 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.495417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fbb1d-f164-4a05-984d-2a052889e919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.495690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fbb1d-f164-4a05-984d-2a052889e919 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.496379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fbb1d-f164-4a05-984d-2a052889e919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.498326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fbb1d-f164-4a05-984d-2a052889e919 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.499842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b82fe-b4e5-4256-81c8-1405e924d5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.500667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b82fe-b4e5-4256-81c8-1405e924d5ad admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.501055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b82fe-b4e5-4256-81c8-1405e924d5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.501434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b82fe-b4e5-4256-81c8-1405e924d5ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.502224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b82fe-b4e5-4256-81c8-1405e924d5ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.504194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b82fe-b4e5-4256-81c8-1405e924d5ad admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.525333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 789/1576] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.526132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fbb1d-f164-4a05-984d-2a052889e919 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.536645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 788/1577] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.536645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b82fe-b4e5-4256-81c8-1405e924d5ad admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.590153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fbb1d-f164-4a05-984d-2a052889e919 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.600207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b82fe-b4e5-4256-81c8-1405e924d5ad admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.605459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4b4fd-a309-4c57-b630-51f6a2df7e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.606123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b4fd-a309-4c57-b630-51f6a2df7e96 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.606415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b4fd-a309-4c57-b630-51f6a2df7e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.606683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b4fd-a309-4c57-b630-51f6a2df7e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.607361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4b4fd-a309-4c57-b630-51f6a2df7e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.609347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4b4fd-a309-4c57-b630-51f6a2df7e96 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.635440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 789/1578] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.636526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4b4fd-a309-4c57-b630-51f6a2df7e96 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.648796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc2ed9-d834-4134-9b8d-dfcfbfcdbd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.649695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc2ed9-d834-4134-9b8d-dfcfbfcdbd81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.649981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc2ed9-d834-4134-9b8d-dfcfbfcdbd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.650262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc2ed9-d834-4134-9b8d-dfcfbfcdbd81 admin admin] PATH_INFO: `/v3/projects/16b90d01b80e450b9357d02c245c9531/users/71ee50293b204bedb1ea03d77b940514/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.665563 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc2ed9-d834-4134-9b8d-dfcfbfcdbd81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b90d01b80e450b9357d02c245c9531, user_id=71ee50293b204bedb1ea03d77b940514, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.666608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: WARNING py.warnings [None req-c2fc2ed9-d834-4134-9b8d-dfcfbfcdbd81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:05:19.666608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: warnings.warn(msg) Jun 08 02:05:19.666608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: Jun 08 02:05:19.668584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc2ed9-d834-4134-9b8d-dfcfbfcdbd81 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.682515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 790/1579] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:19 2022] PUT /identity/v3/projects/16b90d01b80e450b9357d02c245c9531/users/71ee50293b204bedb1ea03d77b940514/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:19.686156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4c599-cab0-4ef0-a2fb-2dbd60eb098b None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.686156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4c599-cab0-4ef0-a2fb-2dbd60eb098b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.686156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4c599-cab0-4ef0-a2fb-2dbd60eb098b None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.686156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 791/1580] 10.222.0.20 () {54 vars in 961 bytes} [Wed Jun 8 02:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:19.686618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4c599-cab0-4ef0-a2fb-2dbd60eb098b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.705332 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4b4fd-a309-4c57-b630-51f6a2df7e96 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.719302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:19.745720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:19.755309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4c599-cab0-4ef0-a2fb-2dbd60eb098b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.760458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966533f-c233-409a-bf4d-9c83740326b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.761127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966533f-c233-409a-bf4d-9c83740326b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.761408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966533f-c233-409a-bf4d-9c83740326b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.761682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966533f-c233-409a-bf4d-9c83740326b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.762361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966533f-c233-409a-bf4d-9c83740326b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.763631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6161971-fee8-4cd6-ac4c-04d30820b992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.764270 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966533f-c233-409a-bf4d-9c83740326b9 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.764421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6161971-fee8-4cd6-ac4c-04d30820b992 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.764607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6161971-fee8-4cd6-ac4c-04d30820b992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.764888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6161971-fee8-4cd6-ac4c-04d30820b992 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.765578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6161971-fee8-4cd6-ac4c-04d30820b992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.767497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6161971-fee8-4cd6-ac4c-04d30820b992 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.785046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 792/1581] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.785833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966533f-c233-409a-bf4d-9c83740326b9 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.787592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 790/1582] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.791750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6161971-fee8-4cd6-ac4c-04d30820b992 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.796048 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966533f-c233-409a-bf4d-9c83740326b9 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.800563 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad9675-909e-4024-8a3c-f8b101dcbfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.801203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad9675-909e-4024-8a3c-f8b101dcbfef admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.801360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6161971-fee8-4cd6-ac4c-04d30820b992 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.801472 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad9675-909e-4024-8a3c-f8b101dcbfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.801752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad9675-909e-4024-8a3c-f8b101dcbfef admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.802424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad9675-909e-4024-8a3c-f8b101dcbfef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.804566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad9675-909e-4024-8a3c-f8b101dcbfef admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.806034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3891c-8fda-499d-bdc5-7b6637d62f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.806680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3891c-8fda-499d-bdc5-7b6637d62f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.806963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3891c-8fda-499d-bdc5-7b6637d62f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.807241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3891c-8fda-499d-bdc5-7b6637d62f58 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.807922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3891c-8fda-499d-bdc5-7b6637d62f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.809866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3891c-8fda-499d-bdc5-7b6637d62f58 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.831377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 791/1583] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/users => generated 292 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.835188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 793/1584] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.835323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3891c-8fda-499d-bdc5-7b6637d62f58 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.835436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad9675-909e-4024-8a3c-f8b101dcbfef admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.844717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3891c-8fda-499d-bdc5-7b6637d62f58 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.898255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad9675-909e-4024-8a3c-f8b101dcbfef admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.901814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcb01c-f7cd-48b1-a239-4f9250c14e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.902515 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcb01c-f7cd-48b1-a239-4f9250c14e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.902728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8110d2-182a-4cd3-845b-7a48c2cacba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.902892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcb01c-f7cd-48b1-a239-4f9250c14e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.903100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcb01c-f7cd-48b1-a239-4f9250c14e56 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.903389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8110d2-182a-4cd3-845b-7a48c2cacba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.903668 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8110d2-182a-4cd3-845b-7a48c2cacba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.903771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcb01c-f7cd-48b1-a239-4f9250c14e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.903935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8110d2-182a-4cd3-845b-7a48c2cacba4 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.904646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8110d2-182a-4cd3-845b-7a48c2cacba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.905694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcb01c-f7cd-48b1-a239-4f9250c14e56 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.906654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8110d2-182a-4cd3-845b-7a48c2cacba4 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.928277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 792/1585] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.930267 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc0b5c-5893-448b-83d2-785c906e0085 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.930556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc0b5c-5893-448b-83d2-785c906e0085 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.930836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc0b5c-5893-448b-83d2-785c906e0085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.934438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 794/1586] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:19.935870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e768558-d604-45f7-8520-550282f1a323 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.936333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e768558-d604-45f7-8520-550282f1a323 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.936641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e768558-d604-45f7-8520-550282f1a323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.949712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1fcc0b5c-5893-448b-83d2-785c906e0085 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:19.950919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1fcc0b5c-5893-448b-83d2-785c906e0085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:19.954521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-2e768558-d604-45f7-8520-550282f1a323 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:19.955694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2e768558-d604-45f7-8520-550282f1a323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:19.958463 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 793/1587] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:19.959364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc0b5c-5893-448b-83d2-785c906e0085 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:19.969738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc0b5c-5893-448b-83d2-785c906e0085 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:19.974958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097628bb-965c-4f45-a6d7-9a2a0c504fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:19.975768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097628bb-965c-4f45-a6d7-9a2a0c504fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:19.976055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097628bb-965c-4f45-a6d7-9a2a0c504fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:19.976333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097628bb-965c-4f45-a6d7-9a2a0c504fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:19.977141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097628bb-965c-4f45-a6d7-9a2a0c504fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:19.978810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097628bb-965c-4f45-a6d7-9a2a0c504fa5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:19.988928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 794/1588] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:19.989727 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097628bb-965c-4f45-a6d7-9a2a0c504fa5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.000147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097628bb-965c-4f45-a6d7-9a2a0c504fa5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.006060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4312a-f4c7-48d4-9bc2-7d1447babfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.007255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4312a-f4c7-48d4-9bc2-7d1447babfbe admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.007728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4312a-f4c7-48d4-9bc2-7d1447babfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.008142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4312a-f4c7-48d4-9bc2-7d1447babfbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.008965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4312a-f4c7-48d4-9bc2-7d1447babfbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.010607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4312a-f4c7-48d4-9bc2-7d1447babfbe admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.023325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 795/1589] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.023325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4312a-f4c7-48d4-9bc2-7d1447babfbe admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.031010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4312a-f4c7-48d4-9bc2-7d1447babfbe admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.031257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 795/1590] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Jun 8 02:05:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:20.031851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e768558-d604-45f7-8520-550282f1a323 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.036050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d8711-e473-4e7e-88b2-7b9a4e141bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.037123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d8711-e473-4e7e-88b2-7b9a4e141bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.037480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d8711-e473-4e7e-88b2-7b9a4e141bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.037857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d8711-e473-4e7e-88b2-7b9a4e141bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.038781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d8711-e473-4e7e-88b2-7b9a4e141bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.040384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d8711-e473-4e7e-88b2-7b9a4e141bf3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.042412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e768558-d604-45f7-8520-550282f1a323 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.046937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fc8c6-9f07-4ead-b6ee-c1241289b700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.047589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fc8c6-9f07-4ead-b6ee-c1241289b700 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.047875 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fc8c6-9f07-4ead-b6ee-c1241289b700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.048152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fc8c6-9f07-4ead-b6ee-c1241289b700 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.048874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fc8c6-9f07-4ead-b6ee-c1241289b700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.050820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fc8c6-9f07-4ead-b6ee-c1241289b700 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.052096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 796/1591] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.052924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d8711-e473-4e7e-88b2-7b9a4e141bf3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.064384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d8711-e473-4e7e-88b2-7b9a4e141bf3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.067034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:20.071094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31639f7d-c310-42c4-9b8c-fb2d767a9387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.071952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31639f7d-c310-42c4-9b8c-fb2d767a9387 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.072355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31639f7d-c310-42c4-9b8c-fb2d767a9387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.072732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31639f7d-c310-42c4-9b8c-fb2d767a9387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.073502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31639f7d-c310-42c4-9b8c-fb2d767a9387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.075056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31639f7d-c310-42c4-9b8c-fb2d767a9387 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.076663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 796/1592] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:20 2022] POST /identity/v3/users => generated 302 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:20.077326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fc8c6-9f07-4ead-b6ee-c1241289b700 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.085879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 797/1593] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.086682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31639f7d-c310-42c4-9b8c-fb2d767a9387 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.086816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fc8c6-9f07-4ead-b6ee-c1241289b700 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.091722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef6d6d-acbb-4d87-b398-ea20ebf3ca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.092380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef6d6d-acbb-4d87-b398-ea20ebf3ca57 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.092690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef6d6d-acbb-4d87-b398-ea20ebf3ca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.092960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef6d6d-acbb-4d87-b398-ea20ebf3ca57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.093690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef6d6d-acbb-4d87-b398-ea20ebf3ca57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.095281 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef6d6d-acbb-4d87-b398-ea20ebf3ca57 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.108499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 797/1594] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.109533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef6d6d-acbb-4d87-b398-ea20ebf3ca57 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.119618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef6d6d-acbb-4d87-b398-ea20ebf3ca57 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.124081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b2b97-8aa9-4679-b021-36010836a4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.124740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b2b97-8aa9-4679-b021-36010836a4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.125020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b2b97-8aa9-4679-b021-36010836a4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.125300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b2b97-8aa9-4679-b021-36010836a4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.126009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b2b97-8aa9-4679-b021-36010836a4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.127579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b2b97-8aa9-4679-b021-36010836a4dc admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.138681 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 798/1595] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.141883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b2b97-8aa9-4679-b021-36010836a4dc admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.144586 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31639f7d-c310-42c4-9b8c-fb2d767a9387 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.149746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d6785-0852-431a-a9bc-5793a2f46261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.150798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d6785-0852-431a-a9bc-5793a2f46261 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.151323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d6785-0852-431a-a9bc-5793a2f46261 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.151835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d6785-0852-431a-a9bc-5793a2f46261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.152501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b2b97-8aa9-4679-b021-36010836a4dc admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.157392 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dc1d4-6e27-42f3-a323-380119ddb791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.158061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dc1d4-6e27-42f3-a323-380119ddb791 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.158344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dc1d4-6e27-42f3-a323-380119ddb791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.158618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dc1d4-6e27-42f3-a323-380119ddb791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.159355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dc1d4-6e27-42f3-a323-380119ddb791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.160163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:20.160972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dc1d4-6e27-42f3-a323-380119ddb791 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.164203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d6785-0852-431a-a9bc-5793a2f46261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.165925 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d6785-0852-431a-a9bc-5793a2f46261 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.172266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 799/1596] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.172995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dc1d4-6e27-42f3-a323-380119ddb791 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.179236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 798/1597] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:20.180555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d6785-0852-431a-a9bc-5793a2f46261 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.186086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dc1d4-6e27-42f3-a323-380119ddb791 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.191616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d6785-0852-431a-a9bc-5793a2f46261 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.192384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654f3b-f37b-466b-9708-2c07a11d7cf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.193080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654f3b-f37b-466b-9708-2c07a11d7cf7 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.193369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654f3b-f37b-466b-9708-2c07a11d7cf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.193681 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654f3b-f37b-466b-9708-2c07a11d7cf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.197764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8129282-a2c9-4df0-a249-3763f45780d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.198305 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8129282-a2c9-4df0-a249-3763f45780d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.198719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8129282-a2c9-4df0-a249-3763f45780d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.199000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8129282-a2c9-4df0-a249-3763f45780d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.199721 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8129282-a2c9-4df0-a249-3763f45780d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.201373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8129282-a2c9-4df0-a249-3763f45780d1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.213791 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 799/1598] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.214628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8129282-a2c9-4df0-a249-3763f45780d1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.224807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8129282-a2c9-4df0-a249-3763f45780d1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.231170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee8f04-9319-4b15-88a2-474e1cb84455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.231835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8f04-9319-4b15-88a2-474e1cb84455 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.232118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8f04-9319-4b15-88a2-474e1cb84455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.232442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee8f04-9319-4b15-88a2-474e1cb84455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.233138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee8f04-9319-4b15-88a2-474e1cb84455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.234726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee8f04-9319-4b15-88a2-474e1cb84455 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.248153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 800/1599] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.249020 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee8f04-9319-4b15-88a2-474e1cb84455 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.261384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee8f04-9319-4b15-88a2-474e1cb84455 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.266409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85654f3b-f37b-466b-9708-2c07a11d7cf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.266928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab5d92-c0d6-40a8-8fc4-cde8f0631a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.267619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab5d92-c0d6-40a8-8fc4-cde8f0631a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.267899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab5d92-c0d6-40a8-8fc4-cde8f0631a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.268170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab5d92-c0d6-40a8-8fc4-cde8f0631a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.268923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab5d92-c0d6-40a8-8fc4-cde8f0631a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.269119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85654f3b-f37b-466b-9708-2c07a11d7cf7 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.270488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab5d92-c0d6-40a8-8fc4-cde8f0631a17 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.279990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 800/1600] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:20.281521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654f3b-f37b-466b-9708-2c07a11d7cf7 service nova] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.282855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 801/1601] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.283869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab5d92-c0d6-40a8-8fc4-cde8f0631a17 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.295352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654f3b-f37b-466b-9708-2c07a11d7cf7 service nova] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.295816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab5d92-c0d6-40a8-8fc4-cde8f0631a17 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.301764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dad4d6-699c-4e1d-9eb9-71503b4c1e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.301969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092d2c5-65ae-4182-bd88-619fd19ecd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.302908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dad4d6-699c-4e1d-9eb9-71503b4c1e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.303160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092d2c5-65ae-4182-bd88-619fd19ecd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.304534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dad4d6-699c-4e1d-9eb9-71503b4c1e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.304675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092d2c5-65ae-4182-bd88-619fd19ecd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.305013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092d2c5-65ae-4182-bd88-619fd19ecd0a admin admin] PATH_INFO: `/v3/projects/27f98152ab3440cf824c9ceb1c790bcf/users/1b43d2b8b9084099b832332a565e741c/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.305574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dad4d6-699c-4e1d-9eb9-71503b4c1e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.306506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dad4d6-699c-4e1d-9eb9-71503b4c1e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.308345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dad4d6-699c-4e1d-9eb9-71503b4c1e67 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.320708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 801/1602] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.321773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dad4d6-699c-4e1d-9eb9-71503b4c1e67 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.325802 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092d2c5-65ae-4182-bd88-619fd19ecd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f98152ab3440cf824c9ceb1c790bcf, user_id=1b43d2b8b9084099b832332a565e741c, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.329092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092d2c5-65ae-4182-bd88-619fd19ecd0a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.335337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dad4d6-699c-4e1d-9eb9-71503b4c1e67 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.344484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4a9da-a290-46bd-a68d-c15b3a18a032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.345772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4a9da-a290-46bd-a68d-c15b3a18a032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.346200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4a9da-a290-46bd-a68d-c15b3a18a032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.346564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4a9da-a290-46bd-a68d-c15b3a18a032 admin admin] PATH_INFO: `/v3/projects/d40d753f66ec4ed2808aa65b825e66f8/users/9e13bd970d2e4d67b62452ae0d6cae56/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.357183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 802/1603] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:20 2022] PUT /identity/v3/projects/27f98152ab3440cf824c9ceb1c790bcf/users/1b43d2b8b9084099b832332a565e741c/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:20.357183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092d2c5-65ae-4182-bd88-619fd19ecd0a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.368046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4a9da-a290-46bd-a68d-c15b3a18a032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40d753f66ec4ed2808aa65b825e66f8, user_id=9e13bd970d2e4d67b62452ae0d6cae56, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.371222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4a9da-a290-46bd-a68d-c15b3a18a032 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.392661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 802/1604] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:20 2022] PUT /identity/v3/projects/d40d753f66ec4ed2808aa65b825e66f8/users/9e13bd970d2e4d67b62452ae0d6cae56/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:20.392661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4a9da-a290-46bd-a68d-c15b3a18a032 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.476825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092d2c5-65ae-4182-bd88-619fd19ecd0a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.485747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ce759-360c-4065-a904-33b61f7755b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.486787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ce759-360c-4065-a904-33b61f7755b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.487092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ce759-360c-4065-a904-33b61f7755b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.487378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ce759-360c-4065-a904-33b61f7755b2 admin admin] PATH_INFO: `/v3/projects/8b762d2e87da463dac84eedfdedbeb7f/users/7ae96cbf198740ec94313c7f0a5f6179/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.489733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4a9da-a290-46bd-a68d-c15b3a18a032 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.496763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165ae58-5457-4f81-ae33-2f6b217e6d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.497450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ae58-5457-4f81-ae33-2f6b217e6d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.497734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ae58-5457-4f81-ae33-2f6b217e6d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.498011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ae58-5457-4f81-ae33-2f6b217e6d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.498742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165ae58-5457-4f81-ae33-2f6b217e6d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.504997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165ae58-5457-4f81-ae33-2f6b217e6d33 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.509778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ce759-360c-4065-a904-33b61f7755b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b762d2e87da463dac84eedfdedbeb7f, user_id=7ae96cbf198740ec94313c7f0a5f6179, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.513397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ce759-360c-4065-a904-33b61f7755b2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.523906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 803/1605] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:20.525011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165ae58-5457-4f81-ae33-2f6b217e6d33 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.544933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 803/1606] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:20 2022] PUT /identity/v3/projects/8b762d2e87da463dac84eedfdedbeb7f/users/7ae96cbf198740ec94313c7f0a5f6179/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:20.545996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ce759-360c-4065-a904-33b61f7755b2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.550836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165ae58-5457-4f81-ae33-2f6b217e6d33 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.612500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:20.655044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ce759-360c-4065-a904-33b61f7755b2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.667856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde16e2b-0d38-4f2b-80b2-4db9771a2812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.670140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde16e2b-0d38-4f2b-80b2-4db9771a2812 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.670657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde16e2b-0d38-4f2b-80b2-4db9771a2812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.671194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde16e2b-0d38-4f2b-80b2-4db9771a2812 admin admin] PATH_INFO: `/v3/projects/988313c714f540718ce907c46ea7acde/users/400b9f835476420289c377bd10f52146/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.675830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96d234-f115-42c7-9f0c-c443d3501d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.676796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96d234-f115-42c7-9f0c-c443d3501d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.677194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96d234-f115-42c7-9f0c-c443d3501d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.677722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96d234-f115-42c7-9f0c-c443d3501d68 admin admin] PATH_INFO: `/v3/projects/b6af942e37894522b760218345ebf027/users/86c9976371bf4594ae14b52d4cc921c9/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.696865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde16e2b-0d38-4f2b-80b2-4db9771a2812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=988313c714f540718ce907c46ea7acde, user_id=400b9f835476420289c377bd10f52146, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.703266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde16e2b-0d38-4f2b-80b2-4db9771a2812 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.709520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96d234-f115-42c7-9f0c-c443d3501d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6af942e37894522b760218345ebf027, user_id=86c9976371bf4594ae14b52d4cc921c9, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.713261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96d234-f115-42c7-9f0c-c443d3501d68 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.726552 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 804/1607] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:20 2022] PUT /identity/v3/projects/988313c714f540718ce907c46ea7acde/users/400b9f835476420289c377bd10f52146/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:20.727944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ef00-3546-4d66-8916-2e08e65dc9b7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.728235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ef00-3546-4d66-8916-2e08e65dc9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.728537 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9ef00-3546-4d66-8916-2e08e65dc9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.731466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 804/1608] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:20 2022] PUT /identity/v3/projects/b6af942e37894522b760218345ebf027/users/86c9976371bf4594ae14b52d4cc921c9/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:20.732995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495f503-1580-4d54-becd-8612c8e93795 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.733330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495f503-1580-4d54-becd-8612c8e93795 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.733612 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495f503-1580-4d54-becd-8612c8e93795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.764247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:20.765413 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-6495f503-1580-4d54-becd-8612c8e93795 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:20.766842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6495f503-1580-4d54-becd-8612c8e93795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:20.771811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6bb9ef00-3546-4d66-8916-2e08e65dc9b7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:20.775099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6bb9ef00-3546-4d66-8916-2e08e65dc9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:20.846243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 805/1609] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:20.848659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495f503-1580-4d54-becd-8612c8e93795 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:20.864892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495f503-1580-4d54-becd-8612c8e93795 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:20.871665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba03188-2ec5-4f05-b074-4be9d3d4ea63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:20.873314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba03188-2ec5-4f05-b074-4be9d3d4ea63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.873497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 805/1610] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:20.873927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba03188-2ec5-4f05-b074-4be9d3d4ea63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.874381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba03188-2ec5-4f05-b074-4be9d3d4ea63 admin admin] PATH_INFO: `/v3/projects/0523a9905e474068b21c44858dd2dc69/users/9d799f35a69648c68ba3de9b7ed721a5/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.875174 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb2ec-8591-4e89-9614-b334fc679466 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.875556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb2ec-8591-4e89-9614-b334fc679466 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.875929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb2ec-8591-4e89-9614-b334fc679466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.895130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba03188-2ec5-4f05-b074-4be9d3d4ea63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0523a9905e474068b21c44858dd2dc69, user_id=9d799f35a69648c68ba3de9b7ed721a5, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:20.899879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba03188-2ec5-4f05-b074-4be9d3d4ea63 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:20.918398 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f9cfb2ec-8591-4e89-9614-b334fc679466 None None] MFA Rules not processed for user `7ae96cbf198740ec94313c7f0a5f6179`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:20.920302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f9cfb2ec-8591-4e89-9614-b334fc679466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:20.927389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 806/1611] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:20 2022] PUT /identity/v3/projects/0523a9905e474068b21c44858dd2dc69/users/9d799f35a69648c68ba3de9b7ed721a5/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:20.942312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdf402-772f-48f2-9ea2-41badcf48928 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.942680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdf402-772f-48f2-9ea2-41badcf48928 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.942888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdf402-772f-48f2-9ea2-41badcf48928 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.944440 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 807/1612] 10.222.0.20 () {54 vars in 912 bytes} [Wed Jun 8 02:05:20 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:20.949974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29381-59b3-4a0e-8a4d-e21976392659 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:20.950448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29381-59b3-4a0e-8a4d-e21976392659 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:20.951134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29381-59b3-4a0e-8a4d-e21976392659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:20.984040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-45b29381-59b3-4a0e-8a4d-e21976392659 None None] MFA Rules not processed for user `400b9f835476420289c377bd10f52146`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:20.984557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-45b29381-59b3-4a0e-8a4d-e21976392659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:21.013681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 806/1613] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:21.014874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dce69-7c2d-4fcd-868e-ac6846af6236 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.015239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dce69-7c2d-4fcd-868e-ac6846af6236 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.015603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606dce69-7c2d-4fcd-868e-ac6846af6236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.051115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-606dce69-7c2d-4fcd-868e-ac6846af6236 None None] MFA Rules not processed for user `86c9976371bf4594ae14b52d4cc921c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:21.053180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-606dce69-7c2d-4fcd-868e-ac6846af6236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:21.073759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 808/1614] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:21.076643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b29381-59b3-4a0e-8a4d-e21976392659 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:21.098418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:21.150488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b29381-59b3-4a0e-8a4d-e21976392659 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:21.152622 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 807/1615] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:21.153897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606dce69-7c2d-4fcd-868e-ac6846af6236 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:21.158035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756cf93-260f-4b07-99c2-de7a59aa62f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:21.158821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756cf93-260f-4b07-99c2-de7a59aa62f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.159120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756cf93-260f-4b07-99c2-de7a59aa62f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.159400 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756cf93-260f-4b07-99c2-de7a59aa62f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.165367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606dce69-7c2d-4fcd-868e-ac6846af6236 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:21.170451 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c0b7d-9682-443a-9b04-3794d1e9ed95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:21.171176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c0b7d-9682-443a-9b04-3794d1e9ed95 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.171527 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c0b7d-9682-443a-9b04-3794d1e9ed95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.171960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c0b7d-9682-443a-9b04-3794d1e9ed95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.198347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:21.233277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756cf93-260f-4b07-99c2-de7a59aa62f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:21.234875 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756cf93-260f-4b07-99c2-de7a59aa62f3 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:21.247910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 809/1616] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:21.248530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c0b7d-9682-443a-9b04-3794d1e9ed95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:21.249311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11074737-0c6b-44b4-bdc2-68ed41aa101e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.249604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11074737-0c6b-44b4-bdc2-68ed41aa101e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.249891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11074737-0c6b-44b4-bdc2-68ed41aa101e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.250381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c0b7d-9682-443a-9b04-3794d1e9ed95 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:21.262511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 808/1617] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:21.264023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490d5fe-7e68-4909-90de-035ec1f3c4d6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.264424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490d5fe-7e68-4909-90de-035ec1f3c4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.264824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490d5fe-7e68-4909-90de-035ec1f3c4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.285661 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a490d5fe-7e68-4909-90de-035ec1f3c4d6 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:21.287612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a490d5fe-7e68-4909-90de-035ec1f3c4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:21.299382 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-11074737-0c6b-44b4-bdc2-68ed41aa101e None None] MFA Rules not processed for user `9d799f35a69648c68ba3de9b7ed721a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:21.301865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-11074737-0c6b-44b4-bdc2-68ed41aa101e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:21.386714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 809/1618] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:05:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:21.388169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4996a-c481-4521-a0a0-2205ff579cde None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.388499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4996a-c481-4521-a0a0-2205ff579cde None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.388782 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4996a-c481-4521-a0a0-2205ff579cde None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.390109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 810/1619] 10.222.0.20 () {54 vars in 961 bytes} [Wed Jun 8 02:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:21.390755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4996a-c481-4521-a0a0-2205ff579cde None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:21.406870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4996a-c481-4521-a0a0-2205ff579cde None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:21.412694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 810/1620] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:21.414379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca7bd9-d20d-4ffd-a427-233d40f90bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:21.415027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a037db1-4868-4d58-93b4-2c49bad6955b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.415190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca7bd9-d20d-4ffd-a427-233d40f90bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.415276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a037db1-4868-4d58-93b4-2c49bad6955b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.415441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca7bd9-d20d-4ffd-a427-233d40f90bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.415720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca7bd9-d20d-4ffd-a427-233d40f90bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.415927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a037db1-4868-4d58-93b4-2c49bad6955b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.416506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca7bd9-d20d-4ffd-a427-233d40f90bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:21.418441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca7bd9-d20d-4ffd-a427-233d40f90bb3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:21.449352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7a037db1-4868-4d58-93b4-2c49bad6955b None None] MFA Rules not processed for user `71ee50293b204bedb1ea03d77b940514`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:21.450947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7a037db1-4868-4d58-93b4-2c49bad6955b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:21.453994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 811/1621] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:21 2022] POST /identity/v3/projects => generated 393 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:21.455104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca7bd9-d20d-4ffd-a427-233d40f90bb3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:21.509063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:21.544289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca7bd9-d20d-4ffd-a427-233d40f90bb3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:21.545672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 811/1622] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:21.546825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16e20-9d91-4c0a-939b-f29b642a3274 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.547118 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16e20-9d91-4c0a-939b-f29b642a3274 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.547401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16e20-9d91-4c0a-939b-f29b642a3274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.577161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-6bd16e20-9d91-4c0a-939b-f29b642a3274 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:21.579323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230644bf-e3b1-4162-a3d8-e1d3a1959c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:21.580283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230644bf-e3b1-4162-a3d8-e1d3a1959c3d service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.580443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6bd16e20-9d91-4c0a-939b-f29b642a3274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:21.580928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230644bf-e3b1-4162-a3d8-e1d3a1959c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.581355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230644bf-e3b1-4162-a3d8-e1d3a1959c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.647800 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:21.671854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230644bf-e3b1-4162-a3d8-e1d3a1959c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:21.673554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230644bf-e3b1-4162-a3d8-e1d3a1959c3d service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:21.685126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 812/1623] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:21.686251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230644bf-e3b1-4162-a3d8-e1d3a1959c3d service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:21.693170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 812/1624] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Jun 8 02:05:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:21.695116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd16e20-9d91-4c0a-939b-f29b642a3274 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:21.696212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230644bf-e3b1-4162-a3d8-e1d3a1959c3d service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:21.701511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b23f9-4b6c-4a31-95ad-3fdfce9d44da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:21.701908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b23f9-4b6c-4a31-95ad-3fdfce9d44da service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.702193 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b23f9-4b6c-4a31-95ad-3fdfce9d44da service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.702705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b23f9-4b6c-4a31-95ad-3fdfce9d44da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.715775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b23f9-4b6c-4a31-95ad-3fdfce9d44da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:21.717429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b23f9-4b6c-4a31-95ad-3fdfce9d44da service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:21.741908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 813/1625] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:21.742861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b23f9-4b6c-4a31-95ad-3fdfce9d44da service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:21.805744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd16e20-9d91-4c0a-939b-f29b642a3274 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:21.813290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91e255-7644-41ae-a442-226ad31116ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:21.813811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91e255-7644-41ae-a442-226ad31116ab admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.814098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91e255-7644-41ae-a442-226ad31116ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.814379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91e255-7644-41ae-a442-226ad31116ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.815087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91e255-7644-41ae-a442-226ad31116ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:21.822974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91e255-7644-41ae-a442-226ad31116ab admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:21.875283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 813/1626] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:21 2022] POST /identity/v3/projects => generated 405 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:21.875283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b3388-80db-4d51-9108-f596205054c7 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:21.875283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b3388-80db-4d51-9108-f596205054c7 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:21.875283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b3388-80db-4d51-9108-f596205054c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:21.899732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b23f9-4b6c-4a31-95ad-3fdfce9d44da service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:21.905468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-cb2b3388-80db-4d51-9108-f596205054c7 None None] MFA Rules not processed for user `71ee50293b204bedb1ea03d77b940514`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:21.906704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-cb2b3388-80db-4d51-9108-f596205054c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:22.043448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f45174-b9de-4398-85be-ee4f5778bdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.048152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f45174-b9de-4398-85be-ee4f5778bdba admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.048492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f45174-b9de-4398-85be-ee4f5778bdba admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.048781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f45174-b9de-4398-85be-ee4f5778bdba admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.049533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f45174-b9de-4398-85be-ee4f5778bdba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.051537 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f45174-b9de-4398-85be-ee4f5778bdba admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.061971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 814/1627] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:22.067265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b3388-80db-4d51-9108-f596205054c7 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.089121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:22.119770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 814/1628] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:21 2022] POST /identity/v3/users => generated 303 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:22.126449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f45174-b9de-4398-85be-ee4f5778bdba admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.160338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f45174-b9de-4398-85be-ee4f5778bdba admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.179632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723364eb-d3c4-487c-a054-2afaf72c3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.180330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723364eb-d3c4-487c-a054-2afaf72c3115 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.181121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723364eb-d3c4-487c-a054-2afaf72c3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.181410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723364eb-d3c4-487c-a054-2afaf72c3115 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.182120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723364eb-d3c4-487c-a054-2afaf72c3115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.184086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723364eb-d3c4-487c-a054-2afaf72c3115 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.211817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 815/1629] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:22 2022] POST /identity/v3/users => generated 309 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:22.213327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723364eb-d3c4-487c-a054-2afaf72c3115 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.233463 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723364eb-d3c4-487c-a054-2afaf72c3115 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.240171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953c2ee-bf02-4c5e-a98c-baef22ab214a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.242879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953c2ee-bf02-4c5e-a98c-baef22ab214a admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.244202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953c2ee-bf02-4c5e-a98c-baef22ab214a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.244513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953c2ee-bf02-4c5e-a98c-baef22ab214a admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.245265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953c2ee-bf02-4c5e-a98c-baef22ab214a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.247160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953c2ee-bf02-4c5e-a98c-baef22ab214a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.259435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 816/1630] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:22.260352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953c2ee-bf02-4c5e-a98c-baef22ab214a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.265662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b3388-80db-4d51-9108-f596205054c7 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.271550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953c2ee-bf02-4c5e-a98c-baef22ab214a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.278748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d69002-4b90-4e7d-977f-706d424a879d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.279965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d69002-4b90-4e7d-977f-706d424a879d service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.281418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d69002-4b90-4e7d-977f-706d424a879d service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.282143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b3f8e-8e93-4a8d-af20-a35e6c3b7fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.282654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d69002-4b90-4e7d-977f-706d424a879d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.284067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b3f8e-8e93-4a8d-af20-a35e6c3b7fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.285102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b3f8e-8e93-4a8d-af20-a35e6c3b7fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.285950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b3f8e-8e93-4a8d-af20-a35e6c3b7fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.287434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b3f8e-8e93-4a8d-af20-a35e6c3b7fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.290075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b3f8e-8e93-4a8d-af20-a35e6c3b7fb7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.317281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 817/1631] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:22.317281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b3f8e-8e93-4a8d-af20-a35e6c3b7fb7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.339977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b3f8e-8e93-4a8d-af20-a35e6c3b7fb7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.355577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70458720-f799-4730-a72f-a38cbabcab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.356512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70458720-f799-4730-a72f-a38cbabcab4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.356799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70458720-f799-4730-a72f-a38cbabcab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.357080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70458720-f799-4730-a72f-a38cbabcab4a admin admin] PATH_INFO: `/v3/projects/6393d327003b4573bd16e88ea5b86aa5/users/13730278f6e34516aa351a42e2e9a363/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.372321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d69002-4b90-4e7d-977f-706d424a879d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.374522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d69002-4b90-4e7d-977f-706d424a879d service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.385341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 815/1632] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:22.386428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d69002-4b90-4e7d-977f-706d424a879d service nova] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.392715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70458720-f799-4730-a72f-a38cbabcab4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6393d327003b4573bd16e88ea5b86aa5, user_id=13730278f6e34516aa351a42e2e9a363, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.395624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70458720-f799-4730-a72f-a38cbabcab4a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.405147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d69002-4b90-4e7d-977f-706d424a879d service nova] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.419239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918426e2-4ecb-4290-89d2-c24516abe86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.420603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918426e2-4ecb-4290-89d2-c24516abe86b admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.431426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 818/1633] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:22 2022] PUT /identity/v3/projects/6393d327003b4573bd16e88ea5b86aa5/users/13730278f6e34516aa351a42e2e9a363/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:22.431601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918426e2-4ecb-4290-89d2-c24516abe86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.431840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918426e2-4ecb-4290-89d2-c24516abe86b admin admin] PATH_INFO: `/v3/projects/073fbec3938845ef83eaaf9e93aea367/users/90dc5062ab7945ebbf430fb58f5a33bb/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.435325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70458720-f799-4730-a72f-a38cbabcab4a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.446576 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:22.467468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918426e2-4ecb-4290-89d2-c24516abe86b admin admin] RBAC: Authorizing `identity:create_grant(project_id=073fbec3938845ef83eaaf9e93aea367, user_id=90dc5062ab7945ebbf430fb58f5a33bb, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.470081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918426e2-4ecb-4290-89d2-c24516abe86b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.494393 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 816/1634] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:22 2022] PUT /identity/v3/projects/073fbec3938845ef83eaaf9e93aea367/users/90dc5062ab7945ebbf430fb58f5a33bb/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:22.498057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918426e2-4ecb-4290-89d2-c24516abe86b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.590057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70458720-f799-4730-a72f-a38cbabcab4a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.669798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918426e2-4ecb-4290-89d2-c24516abe86b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.687509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9dc87-41d9-4f34-9027-0bb7386c60c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.688613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9dc87-41d9-4f34-9027-0bb7386c60c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.689133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9dc87-41d9-4f34-9027-0bb7386c60c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.689644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9dc87-41d9-4f34-9027-0bb7386c60c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.690600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9dc87-41d9-4f34-9027-0bb7386c60c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.692471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9dc87-41d9-4f34-9027-0bb7386c60c2 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.712001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 817/1635] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:22.713171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9dc87-41d9-4f34-9027-0bb7386c60c2 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.752008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60d6cf-09b2-4c75-9513-cc42833c13a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.752625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:22.754964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60d6cf-09b2-4c75-9513-cc42833c13a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.755646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60d6cf-09b2-4c75-9513-cc42833c13a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.756054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60d6cf-09b2-4c75-9513-cc42833c13a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.757514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60d6cf-09b2-4c75-9513-cc42833c13a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.760415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60d6cf-09b2-4c75-9513-cc42833c13a5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.776010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 819/1636] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:22.777530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60d6cf-09b2-4c75-9513-cc42833c13a5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.794243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60d6cf-09b2-4c75-9513-cc42833c13a5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.799465 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13fe91-f81f-4d10-a37a-81ff28ce37da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.800681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13fe91-f81f-4d10-a37a-81ff28ce37da admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.801096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13fe91-f81f-4d10-a37a-81ff28ce37da admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.801472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13fe91-f81f-4d10-a37a-81ff28ce37da admin admin] PATH_INFO: `/v3/projects/073fbec3938845ef83eaaf9e93aea367/users/90dc5062ab7945ebbf430fb58f5a33bb/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.804942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13fe91-f81f-4d10-a37a-81ff28ce37da admin admin] RBAC: Authorizing `identity:create_grant(project_id=073fbec3938845ef83eaaf9e93aea367, user_id=90dc5062ab7945ebbf430fb58f5a33bb, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.808946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13fe91-f81f-4d10-a37a-81ff28ce37da admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.837248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 820/1637] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:22 2022] PUT /identity/v3/projects/073fbec3938845ef83eaaf9e93aea367/users/90dc5062ab7945ebbf430fb58f5a33bb/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:22.838510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13fe91-f81f-4d10-a37a-81ff28ce37da admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:22.876084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9dc87-41d9-4f34-9027-0bb7386c60c2 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.948549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13fe91-f81f-4d10-a37a-81ff28ce37da admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:22.956173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c263dd9-05d9-43de-bdba-354174dfa198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:22.958487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c263dd9-05d9-43de-bdba-354174dfa198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.958955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c263dd9-05d9-43de-bdba-354174dfa198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.959332 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c263dd9-05d9-43de-bdba-354174dfa198 admin admin] PATH_INFO: `/v3/projects/6393d327003b4573bd16e88ea5b86aa5/users/13730278f6e34516aa351a42e2e9a363/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.963559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c263dd9-05d9-43de-bdba-354174dfa198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6393d327003b4573bd16e88ea5b86aa5, user_id=13730278f6e34516aa351a42e2e9a363, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:22.967902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c263dd9-05d9-43de-bdba-354174dfa198 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:22.989425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 821/1638] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:22 2022] PUT /identity/v3/projects/6393d327003b4573bd16e88ea5b86aa5/users/13730278f6e34516aa351a42e2e9a363/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:22.991884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef9019-9d5e-4217-b1f2-7a107e66b1ed None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.992285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef9019-9d5e-4217-b1f2-7a107e66b1ed None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.992656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef9019-9d5e-4217-b1f2-7a107e66b1ed None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:22.994482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 822/1639] 10.222.0.20 () {54 vars in 912 bytes} [Wed Jun 8 02:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:22.995605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657d050-5870-4fce-8dc9-207c0d85a295 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:22.996367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657d050-5870-4fce-8dc9-207c0d85a295 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:22.996367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657d050-5870-4fce-8dc9-207c0d85a295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:23.039642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647c508-646d-4cc9-a9be-5c4731a95127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:23.040370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647c508-646d-4cc9-a9be-5c4731a95127 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:23.040831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647c508-646d-4cc9-a9be-5c4731a95127 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:23.041145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647c508-646d-4cc9-a9be-5c4731a95127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:23.043018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0657d050-5870-4fce-8dc9-207c0d85a295 None None] MFA Rules not processed for user `90dc5062ab7945ebbf430fb58f5a33bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:23.048348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0657d050-5870-4fce-8dc9-207c0d85a295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:23.070935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:23.133290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647c508-646d-4cc9-a9be-5c4731a95127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:23.138405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647c508-646d-4cc9-a9be-5c4731a95127 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:23.159264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 818/1640] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:23.163964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beea5f-19a3-4804-8d25-92b500ca15d8 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:23.164354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beea5f-19a3-4804-8d25-92b500ca15d8 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:23.164669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beea5f-19a3-4804-8d25-92b500ca15d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:23.176789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 823/1641] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:22 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:23.187779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acb559-8c7b-4d65-846a-49e721249db4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:23.188150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acb559-8c7b-4d65-846a-49e721249db4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:23.188460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acb559-8c7b-4d65-846a-49e721249db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:23.213670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-37acb559-8c7b-4d65-846a-49e721249db4 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:23.214971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-37acb559-8c7b-4d65-846a-49e721249db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:23.232201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-72beea5f-19a3-4804-8d25-92b500ca15d8 None None] MFA Rules not processed for user `13730278f6e34516aa351a42e2e9a363`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:23.233491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-72beea5f-19a3-4804-8d25-92b500ca15d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:23.339758 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 824/1642] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:05:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:23.341137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acb559-8c7b-4d65-846a-49e721249db4 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:23.347296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:23.415410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 819/1643] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:23.416286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beea5f-19a3-4804-8d25-92b500ca15d8 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:23.476342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acb559-8c7b-4d65-846a-49e721249db4 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:23.480201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7beb3-03c2-4fe2-8049-005db3935841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:23.480894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7beb3-03c2-4fe2-8049-005db3935841 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:23.481182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7beb3-03c2-4fe2-8049-005db3935841 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:23.481459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7beb3-03c2-4fe2-8049-005db3935841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:23.495047 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beea5f-19a3-4804-8d25-92b500ca15d8 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:23.499028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510cb6a-f066-4623-a903-4dddff552a46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:23.499686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510cb6a-f066-4623-a903-4dddff552a46 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:23.499977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510cb6a-f066-4623-a903-4dddff552a46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:23.500255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510cb6a-f066-4623-a903-4dddff552a46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:23.502196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7beb3-03c2-4fe2-8049-005db3935841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:23.503803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7beb3-03c2-4fe2-8049-005db3935841 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:23.513717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510cb6a-f066-4623-a903-4dddff552a46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:23.515931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510cb6a-f066-4623-a903-4dddff552a46 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:23.518922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 825/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:23 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:23.530679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 820/1645] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:23.584247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7beb3-03c2-4fe2-8049-005db3935841 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:23.595676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7beb3-03c2-4fe2-8049-005db3935841 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:23.606410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b69dd-47d0-43ea-811c-971e853909d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:23.608146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b69dd-47d0-43ea-811c-971e853909d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:23.608449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b69dd-47d0-43ea-811c-971e853909d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:23.608738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b69dd-47d0-43ea-811c-971e853909d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:23.634142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b69dd-47d0-43ea-811c-971e853909d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:23.637117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b69dd-47d0-43ea-811c-971e853909d6 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:23.661987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 826/1646] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:23.808938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f895c5e-db11-4fcd-a808-bcc6fafb1fda None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:23.809052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f895c5e-db11-4fcd-a808-bcc6fafb1fda None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:23.809319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f895c5e-db11-4fcd-a808-bcc6fafb1fda None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:23.810818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b69dd-47d0-43ea-811c-971e853909d6 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:23.810968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 821/1647] 10.222.0.20 () {54 vars in 912 bytes} [Wed Jun 8 02:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:23.819938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51403aaf-b119-4891-a5b8-d2b19a03fb41 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:23.820268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51403aaf-b119-4891-a5b8-d2b19a03fb41 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:23.820534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51403aaf-b119-4891-a5b8-d2b19a03fb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:23.823166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b69dd-47d0-43ea-811c-971e853909d6 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:23.832613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5567fed-4b92-4d07-99f2-b3a8229f7d74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:23.833248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5567fed-4b92-4d07-99f2-b3a8229f7d74 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:23.833497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5567fed-4b92-4d07-99f2-b3a8229f7d74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:23.833774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5567fed-4b92-4d07-99f2-b3a8229f7d74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:23.905484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-51403aaf-b119-4891-a5b8-d2b19a03fb41 None None] MFA Rules not processed for user `d26f4426ba8c43399c4a7ef2f1701f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:23.910062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-51403aaf-b119-4891-a5b8-d2b19a03fb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:23.925614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:24.042001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 822/1648] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:05:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:24.047792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5567fed-4b92-4d07-99f2-b3a8229f7d74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:24.053627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5567fed-4b92-4d07-99f2-b3a8229f7d74 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:24.090164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 827/1649] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:24.110406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51403aaf-b119-4891-a5b8-d2b19a03fb41 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:24.134187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51403aaf-b119-4891-a5b8-d2b19a03fb41 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:24.145959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41465051-7dbc-4b01-80cb-2331b401cb4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:24.146702 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41465051-7dbc-4b01-80cb-2331b401cb4e service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:24.147014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41465051-7dbc-4b01-80cb-2331b401cb4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:24.147503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41465051-7dbc-4b01-80cb-2331b401cb4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:24.166503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41465051-7dbc-4b01-80cb-2331b401cb4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:24.168807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41465051-7dbc-4b01-80cb-2331b401cb4e service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:24.191680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 823/1650] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:24.394403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5567fed-4b92-4d07-99f2-b3a8229f7d74 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:24.513159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5567fed-4b92-4d07-99f2-b3a8229f7d74 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:24.527926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:24.528865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:24.529152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:24.529424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] PATH_INFO: `/v3/users/9d799f35a69648c68ba3de9b7ed721a5` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:24.530701 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d799f35a69648c68ba3de9b7ed721a5)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:24.532801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:24.642158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d799f35a69648c68ba3de9b7ed721a5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:24.642854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d799f35a69648c68ba3de9b7ed721a5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:24.664326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d799f35a69648c68ba3de9b7ed721a5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:24.665217 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d799f35a69648c68ba3de9b7ed721a5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:24.683511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d799f35a69648c68ba3de9b7ed721a5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:24.724789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 828/1651] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:24 2022] DELETE /identity/v3/users/9d799f35a69648c68ba3de9b7ed721a5 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:24.726897 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52523b7-89e8-44fb-82d2-33b576145757 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:24.728062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52523b7-89e8-44fb-82d2-33b576145757 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:24.728540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52523b7-89e8-44fb-82d2-33b576145757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:24.814059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b52523b7-89e8-44fb-82d2-33b576145757 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:24.814059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b52523b7-89e8-44fb-82d2-33b576145757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:24.992268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 824/1652] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:25.051786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:25.059173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:25.206595 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86159f3-91bf-40ee-bd1c-1b7a5b1eb737 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:25.211633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aed2f-c6dc-4c93-aa1c-d422ada97319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:25.212308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aed2f-c6dc-4c93-aa1c-d422ada97319 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:25.217017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aed2f-c6dc-4c93-aa1c-d422ada97319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:25.217017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aed2f-c6dc-4c93-aa1c-d422ada97319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:25.235257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:25.244062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aed2f-c6dc-4c93-aa1c-d422ada97319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:25.245910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aed2f-c6dc-4c93-aa1c-d422ada97319 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:25.295976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 829/1653] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:25.305772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6246bcf-4378-46e1-8999-00ccc1af14da None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:25.306114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6246bcf-4378-46e1-8999-00ccc1af14da None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:25.306401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6246bcf-4378-46e1-8999-00ccc1af14da None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:25.307842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 825/1654] 10.222.0.20 () {54 vars in 960 bytes} [Wed Jun 8 02:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:25.316055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432269f7-d2ec-41a6-99ac-cb7b1d1738d7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:25.316367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432269f7-d2ec-41a6-99ac-cb7b1d1738d7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:25.316696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432269f7-d2ec-41a6-99ac-cb7b1d1738d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:25.360194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-432269f7-d2ec-41a6-99ac-cb7b1d1738d7 None None] MFA Rules not processed for user `4eb69af145154c6d85bd4b845979e6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:25.363129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-432269f7-d2ec-41a6-99ac-cb7b1d1738d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:25.540434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 830/1655] 10.222.0.20 () {58 vars in 1057 bytes} [Wed Jun 8 02:05:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:25.550868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6246bcf-4378-46e1-8999-00ccc1af14da None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:25.561647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:25.579948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6246bcf-4378-46e1-8999-00ccc1af14da None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:25.595872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb57a2-33c0-46e0-aadf-920a9d374ba9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:25.595872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb57a2-33c0-46e0-aadf-920a9d374ba9 service placement] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:25.595872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb57a2-33c0-46e0-aadf-920a9d374ba9 service placement] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:25.595872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb57a2-33c0-46e0-aadf-920a9d374ba9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:25.596746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-31cb57a2-33c0-46e0-aadf-920a9d374ba9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:25.610411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb57a2-33c0-46e0-aadf-920a9d374ba9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:25.612699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb57a2-33c0-46e0-aadf-920a9d374ba9 service placement] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:25.648624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 826/1656] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:26.515144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432269f7-d2ec-41a6-99ac-cb7b1d1738d7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:26.516748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-432269f7-d2ec-41a6-99ac-cb7b1d1738d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:26.554783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432269f7-d2ec-41a6-99ac-cb7b1d1738d7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:26.567289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:26.568294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:26.571547 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:26.572163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] PATH_INFO: `/v3/projects/0523a9905e474068b21c44858dd2dc69` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:26.577392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0523a9905e474068b21c44858dd2dc69)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:26.578624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: WARNING py.warnings [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:05:26.578624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: warnings.warn(msg) Jun 08 02:05:26.578624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: Jun 08 02:05:26.580196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:26.751464 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0523a9905e474068b21c44858dd2dc69', 'request_id': 'req-24d2973b-04d3-4a2f-aad3-316c260ca0d7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:26.770028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0523a9905e474068b21c44858dd2dc69', 'request_id': 'req-24d2973b-04d3-4a2f-aad3-316c260ca0d7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:26.792695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0523a9905e474068b21c44858dd2dc69', 'request_id': 'req-24d2973b-04d3-4a2f-aad3-316c260ca0d7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:26.823804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] The token cache is being invalidate because project 0523a9905e474068b21c44858dd2dc69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:26.824271 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:26.830090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:26.832422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 831/1657] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:26 2022] DELETE /identity/v3/projects/0523a9905e474068b21c44858dd2dc69 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:26.837852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb57a2-33c0-46e0-aadf-920a9d374ba9 service placement] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:26.920946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb57a2-33c0-46e0-aadf-920a9d374ba9 service placement] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:26.922233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:26.925995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:27.229605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2973b-04d3-4a2f-aad3-316c260ca0d7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:27.266045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:27.266383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:27.267481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:27.267481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] PATH_INFO: `/v3/users/fdf621d5692b4fe9995b026819e71f92` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:27.268431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdf621d5692b4fe9995b026819e71f92)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:27.277139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:27.291261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:27.294367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:27.297539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:27.300629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] PATH_INFO: `/v3/users/90dc5062ab7945ebbf430fb58f5a33bb` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:27.305242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90dc5062ab7945ebbf430fb58f5a33bb)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:27.307629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: WARNING py.warnings [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 08 02:05:27.307629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: warnings.warn(msg) Jun 08 02:05:27.307629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: Jun 08 02:05:27.309895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:27.339992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:27.444337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdf621d5692b4fe9995b026819e71f92'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:27.445366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdf621d5692b4fe9995b026819e71f92'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:27.471498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdf621d5692b4fe9995b026819e71f92'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:27.473565 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdf621d5692b4fe9995b026819e71f92'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:27.501389 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-2fc26717-f0c8-4741-b478-8a9cff770427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdf621d5692b4fe9995b026819e71f92'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:27.513756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90dc5062ab7945ebbf430fb58f5a33bb'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:27.515670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90dc5062ab7945ebbf430fb58f5a33bb'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:27.548632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 832/1658] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:26 2022] DELETE /identity/v3/users/fdf621d5692b4fe9995b026819e71f92 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:27.552849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90dc5062ab7945ebbf430fb58f5a33bb'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:27.555129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f1c9-48a9-400f-a0b2-e2701982e39c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:27.555624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f1c9-48a9-400f-a0b2-e2701982e39c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:27.556255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f1c9-48a9-400f-a0b2-e2701982e39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:27.559193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90dc5062ab7945ebbf430fb58f5a33bb'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:27.583508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-56d4f1c9-48a9-400f-a0b2-e2701982e39c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:27.585509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-56d4f1c9-48a9-400f-a0b2-e2701982e39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:27.609139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90dc5062ab7945ebbf430fb58f5a33bb'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:27.639736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 827/1659] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:26 2022] DELETE /identity/v3/users/90dc5062ab7945ebbf430fb58f5a33bb => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:27.701103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 833/1660] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:27.748489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:27.750186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:27.886111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa6d36-a5a6-407a-9a71-d3b9ba34e551 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:27.893131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4c85f-5020-4fb1-b0aa-03a83fc0dffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:27.893819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4c85f-5020-4fb1-b0aa-03a83fc0dffd service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:27.894106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4c85f-5020-4fb1-b0aa-03a83fc0dffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:27.894516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4c85f-5020-4fb1-b0aa-03a83fc0dffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:27.967022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:28.017242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4c85f-5020-4fb1-b0aa-03a83fc0dffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:28.018775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4c85f-5020-4fb1-b0aa-03a83fc0dffd service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:28.030909 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 828/1661] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:29.273344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4f1c9-48a9-400f-a0b2-e2701982e39c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:29.274517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-56d4f1c9-48a9-400f-a0b2-e2701982e39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:29.300991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4f1c9-48a9-400f-a0b2-e2701982e39c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:29.313688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:29.317716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:29.318205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:29.321310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] PATH_INFO: `/v3/projects/073fbec3938845ef83eaaf9e93aea367` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:29.323094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=073fbec3938845ef83eaaf9e93aea367)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:29.327155 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:29.516056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '073fbec3938845ef83eaaf9e93aea367', 'request_id': 'req-6ecf479d-074b-4e84-9db6-0a88e1292e2a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:29.565607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '073fbec3938845ef83eaaf9e93aea367', 'request_id': 'req-6ecf479d-074b-4e84-9db6-0a88e1292e2a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:29.586465 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '073fbec3938845ef83eaaf9e93aea367', 'request_id': 'req-6ecf479d-074b-4e84-9db6-0a88e1292e2a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:29.618810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] The token cache is being invalidate because project 073fbec3938845ef83eaaf9e93aea367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:29.620239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:29.622634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ecf479d-074b-4e84-9db6-0a88e1292e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:29.627878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 834/1662] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:29 2022] DELETE /identity/v3/projects/073fbec3938845ef83eaaf9e93aea367 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:29.632932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b5f6-b88d-41bf-bf39-f1e0a76f9b71 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:29.634528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b5f6-b88d-41bf-bf39-f1e0a76f9b71 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:29.634989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b5f6-b88d-41bf-bf39-f1e0a76f9b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:29.671701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-08f5b5f6-b88d-41bf-bf39-f1e0a76f9b71 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:29.684207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-08f5b5f6-b88d-41bf-bf39-f1e0a76f9b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:29.868486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 829/1663] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:29.871832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f3386-3e8a-41ca-952f-2f63e58b067c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:29.872138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f3386-3e8a-41ca-952f-2f63e58b067c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:29.872442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f3386-3e8a-41ca-952f-2f63e58b067c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:29.935111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-665f3386-3e8a-41ca-952f-2f63e58b067c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:29.936500 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-665f3386-3e8a-41ca-952f-2f63e58b067c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:29.970874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 835/1664] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:29.974557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5b5f6-b88d-41bf-bf39-f1e0a76f9b71 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:29.992142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5b5f6-b88d-41bf-bf39-f1e0a76f9b71 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:30.018660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50b2e8-c705-44d1-bca8-63c491e8bb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:30.020299 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50b2e8-c705-44d1-bca8-63c491e8bb96 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:30.022767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50b2e8-c705-44d1-bca8-63c491e8bb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:30.024079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50b2e8-c705-44d1-bca8-63c491e8bb96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:30.025086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50b2e8-c705-44d1-bca8-63c491e8bb96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:30.026788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50b2e8-c705-44d1-bca8-63c491e8bb96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:30.033201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50b2e8-c705-44d1-bca8-63c491e8bb96 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:30.120694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 830/1665] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:30.122634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f3386-3e8a-41ca-952f-2f63e58b067c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:30.144215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f3386-3e8a-41ca-952f-2f63e58b067c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:30.165170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fa471-02ee-4eab-aa16-9b1224187eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:30.166962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fa471-02ee-4eab-aa16-9b1224187eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:30.167250 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fa471-02ee-4eab-aa16-9b1224187eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:30.167532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fa471-02ee-4eab-aa16-9b1224187eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:30.173403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fa471-02ee-4eab-aa16-9b1224187eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:30.183704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fa471-02ee-4eab-aa16-9b1224187eaf admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:30.196627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:30.245744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 836/1666] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:30 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:30.250830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50b2e8-c705-44d1-bca8-63c491e8bb96 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:30.279489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50b2e8-c705-44d1-bca8-63c491e8bb96 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:30.318534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5abbf-0281-47bb-859e-d8436305022f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:30.320248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5abbf-0281-47bb-859e-d8436305022f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:30.320868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5abbf-0281-47bb-859e-d8436305022f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:30.363611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-11c5abbf-0281-47bb-859e-d8436305022f None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:30.365213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-11c5abbf-0281-47bb-859e-d8436305022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:30.510742 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 837/1667] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:05:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:30.528168 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5abbf-0281-47bb-859e-d8436305022f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:30.529927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-11c5abbf-0281-47bb-859e-d8436305022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:30.545161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997388cd-1c84-406e-a34b-e9316057fa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:30.545973 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997388cd-1c84-406e-a34b-e9316057fa65 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:30.546263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997388cd-1c84-406e-a34b-e9316057fa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:30.546616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997388cd-1c84-406e-a34b-e9316057fa65 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:30.547352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997388cd-1c84-406e-a34b-e9316057fa65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:30.549536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997388cd-1c84-406e-a34b-e9316057fa65 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:30.613914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 831/1668] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:30 2022] POST /identity/v3/users => generated 293 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:30.618895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997388cd-1c84-406e-a34b-e9316057fa65 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:30.643570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997388cd-1c84-406e-a34b-e9316057fa65 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:30.658353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10d943-ae25-4030-8e68-319c68ebac47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:30.659112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10d943-ae25-4030-8e68-319c68ebac47 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:30.659406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10d943-ae25-4030-8e68-319c68ebac47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:30.659686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10d943-ae25-4030-8e68-319c68ebac47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:30.660615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10d943-ae25-4030-8e68-319c68ebac47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:30.662673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10d943-ae25-4030-8e68-319c68ebac47 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:30.701830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 832/1669] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:05:30.714887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10d943-ae25-4030-8e68-319c68ebac47 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:30.755203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5abbf-0281-47bb-859e-d8436305022f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:30.764104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10d943-ae25-4030-8e68-319c68ebac47 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:30.773720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:30.774629 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:30.774916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:30.775197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] PATH_INFO: `/v3/users/400b9f835476420289c377bd10f52146` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:30.781973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] RBAC: Authorizing `identity:delete_user(user_id=400b9f835476420289c377bd10f52146)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:30.786492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c2aa7-4501-4593-961d-36f29820780c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:30.788860 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:30.795282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c2aa7-4501-4593-961d-36f29820780c admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:30.796427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c2aa7-4501-4593-961d-36f29820780c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:30.796858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c2aa7-4501-4593-961d-36f29820780c admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:30.799342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c2aa7-4501-4593-961d-36f29820780c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:30.801228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c2aa7-4501-4593-961d-36f29820780c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:30.837185 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 833/1670] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:30.841200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c2aa7-4501-4593-961d-36f29820780c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:30.872009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-268c2aa7-4501-4593-961d-36f29820780c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:30.938437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c2aa7-4501-4593-961d-36f29820780c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:30.954791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:30.957495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:30.957796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:30.958884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] PATH_INFO: `/v3/projects/c1f3f7571cb24d069d7bb1182bafb300` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:30.966190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f3f7571cb24d069d7bb1182bafb300)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:30.979816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:31.003345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:31.072699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400b9f835476420289c377bd10f52146'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:31.073869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400b9f835476420289c377bd10f52146'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:31.088066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400b9f835476420289c377bd10f52146'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:31.088843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400b9f835476420289c377bd10f52146'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:31.104852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f3f7571cb24d069d7bb1182bafb300', 'request_id': 'req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:31.123424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400b9f835476420289c377bd10f52146'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:31.132493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f3f7571cb24d069d7bb1182bafb300', 'request_id': 'req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:31.168593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f3f7571cb24d069d7bb1182bafb300', 'request_id': 'req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:31.187842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 838/1671] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:30 2022] DELETE /identity/v3/users/400b9f835476420289c377bd10f52146 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:05:31.189314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:31.198942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:31.218196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] The token cache is being invalidate because project c1f3f7571cb24d069d7bb1182bafb300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:31.218507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:31.219357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:31.222450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 834/1672] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:30 2022] DELETE /identity/v3/projects/c1f3f7571cb24d069d7bb1182bafb300 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:31.223118 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:31.226305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:31.239431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:31.369563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f7ade-4d31-4ec5-b466-0de4ce1ed23c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:31.385137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33fad4-15a7-4c6c-ac08-8e48a93408f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:31.386374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33fad4-15a7-4c6c-ac08-8e48a93408f5 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:31.386914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33fad4-15a7-4c6c-ac08-8e48a93408f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:31.387381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33fad4-15a7-4c6c-ac08-8e48a93408f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:31.395417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ed33fad4-15a7-4c6c-ac08-8e48a93408f5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:31.406983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9f358-02e6-4904-8b83-4a6f2a30d3b6 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:31.554989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33fad4-15a7-4c6c-ac08-8e48a93408f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:31.556812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33fad4-15a7-4c6c-ac08-8e48a93408f5 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:31.578966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 839/1673] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:05:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:05:31.580326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02427e2-0087-4c4c-a084-5bcaab29c513 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:31.580641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02427e2-0087-4c4c-a084-5bcaab29c513 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:31.580917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02427e2-0087-4c4c-a084-5bcaab29c513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:31.605927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c5896-e03a-48d3-a2f3-dc2d83dd9b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:31.609003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c5896-e03a-48d3-a2f3-dc2d83dd9b88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:31.610282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c5896-e03a-48d3-a2f3-dc2d83dd9b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:31.611052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c5896-e03a-48d3-a2f3-dc2d83dd9b88 admin admin] PATH_INFO: `/v3/projects/5092352dc53c4014a067160b260b2132/users/7725e1370a3e4e3f8e19fc6a00ee7a64/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:31.653503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e02427e2-0087-4c4c-a084-5bcaab29c513 None None] MFA Rules not processed for user `1b43d2b8b9084099b832332a565e741c`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:31.655182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e02427e2-0087-4c4c-a084-5bcaab29c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:31.665629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c5896-e03a-48d3-a2f3-dc2d83dd9b88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5092352dc53c4014a067160b260b2132, user_id=7725e1370a3e4e3f8e19fc6a00ee7a64, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:31.693188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c5896-e03a-48d3-a2f3-dc2d83dd9b88 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:31.726634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 835/1674] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:31 2022] PUT /identity/v3/projects/5092352dc53c4014a067160b260b2132/users/7725e1370a3e4e3f8e19fc6a00ee7a64/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:31.732676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730c42c-865c-499a-b47c-e26cba2f7739 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:31.732999 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730c42c-865c-499a-b47c-e26cba2f7739 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:31.733285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730c42c-865c-499a-b47c-e26cba2f7739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:31.788975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-4730c42c-865c-499a-b47c-e26cba2f7739 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:31.790277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4730c42c-865c-499a-b47c-e26cba2f7739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:31.797059 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 840/1675] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:31.798947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb3283-826a-46d6-9169-e93778f862a0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:31.800300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb3283-826a-46d6-9169-e93778f862a0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:31.800817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb3283-826a-46d6-9169-e93778f862a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:31.862361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:31.874867 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-62bb3283-826a-46d6-9169-e93778f862a0 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:31.883181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-62bb3283-826a-46d6-9169-e93778f862a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:31.984754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 836/1676] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:31.986328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc534b-2866-4f2b-992d-24664b4e7178 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:31.986627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc534b-2866-4f2b-992d-24664b4e7178 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:31.986913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc534b-2866-4f2b-992d-24664b4e7178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:32.042368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:32.057473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-f1fc534b-2866-4f2b-992d-24664b4e7178 None None] MFA Rules not processed for user `7725e1370a3e4e3f8e19fc6a00ee7a64`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:32.058770 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f1fc534b-2866-4f2b-992d-24664b4e7178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:32.096723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 841/1677] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:32.097888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb3283-826a-46d6-9169-e93778f862a0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:32.099534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-62bb3283-826a-46d6-9169-e93778f862a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:32.268549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 837/1678] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:32.273303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc534b-2866-4f2b-992d-24664b4e7178 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:32.276034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f1fc534b-2866-4f2b-992d-24664b4e7178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:32.363145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb3283-826a-46d6-9169-e93778f862a0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:32.384344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3ce7d-10cd-4f76-9ddd-7ac80c69b974 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:32.385163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3ce7d-10cd-4f76-9ddd-7ac80c69b974 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:32.385400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3ce7d-10cd-4f76-9ddd-7ac80c69b974 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:32.385680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3ce7d-10cd-4f76-9ddd-7ac80c69b974 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:32.502109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc534b-2866-4f2b-992d-24664b4e7178 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:32.521696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4766902-cb8a-42d7-a567-a3f25d798937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:32.522964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766902-cb8a-42d7-a567-a3f25d798937 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:32.523395 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766902-cb8a-42d7-a567-a3f25d798937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:32.523830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766902-cb8a-42d7-a567-a3f25d798937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:32.548231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4766902-cb8a-42d7-a567-a3f25d798937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:32.550194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4766902-cb8a-42d7-a567-a3f25d798937 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:32.577004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 838/1679] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:32.583462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d8b8d-d49c-4229-ba66-3edba0cc71f1 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:32.583735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d8b8d-d49c-4229-ba66-3edba0cc71f1 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:32.585235 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d8b8d-d49c-4229-ba66-3edba0cc71f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:32.633174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-254d8b8d-d49c-4229-ba66-3edba0cc71f1 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:32.645946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3ce7d-10cd-4f76-9ddd-7ac80c69b974 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:32.649515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-254d8b8d-d49c-4229-ba66-3edba0cc71f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:32.654125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3ce7d-10cd-4f76-9ddd-7ac80c69b974 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:32.668450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 839/1680] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:32.668450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d8b8d-d49c-4229-ba66-3edba0cc71f1 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:32.680599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 842/1681] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:32.681444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3ce7d-10cd-4f76-9ddd-7ac80c69b974 service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:32.682879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2eb3ce7d-10cd-4f76-9ddd-7ac80c69b974 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:32.684871 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d8b8d-d49c-4229-ba66-3edba0cc71f1 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:32.689824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4dd0-eeb1-4385-bc82-22618371a4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:32.690503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4dd0-eeb1-4385-bc82-22618371a4af admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:32.690789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4dd0-eeb1-4385-bc82-22618371a4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:32.691066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf4dd0-eeb1-4385-bc82-22618371a4af admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:32.692045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf4dd0-eeb1-4385-bc82-22618371a4af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:32.694093 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf4dd0-eeb1-4385-bc82-22618371a4af admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:32.699310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3ce7d-10cd-4f76-9ddd-7ac80c69b974 service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:32.704067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:32.705239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:32.705526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:32.705797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] PATH_INFO: `/v3/users/7ae96cbf198740ec94313c7f0a5f6179` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:32.707064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae96cbf198740ec94313c7f0a5f6179)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:32.710091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:32.755160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 840/1682] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:32 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:32.756231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4dd0-eeb1-4385-bc82-22618371a4af admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:32.842639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:32.880742 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae96cbf198740ec94313c7f0a5f6179'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:32.884034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae96cbf198740ec94313c7f0a5f6179'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:32.918827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf4dd0-eeb1-4385-bc82-22618371a4af admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:32.924919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56d46d-cc63-4504-87fc-5bb2806422c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:32.925619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56d46d-cc63-4504-87fc-5bb2806422c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:32.925906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56d46d-cc63-4504-87fc-5bb2806422c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:32.926187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56d46d-cc63-4504-87fc-5bb2806422c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:32.926928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56d46d-cc63-4504-87fc-5bb2806422c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:32.936618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56d46d-cc63-4504-87fc-5bb2806422c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:32.940437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56d46d-cc63-4504-87fc-5bb2806422c1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:32.941208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae96cbf198740ec94313c7f0a5f6179'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:32.945670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae96cbf198740ec94313c7f0a5f6179'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:32.977202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae96cbf198740ec94313c7f0a5f6179'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:33.005601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:33.046089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 841/1683] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:33.046925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56d46d-cc63-4504-87fc-5bb2806422c1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:33.049938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0d56d46d-cc63-4504-87fc-5bb2806422c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:33.056275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 843/1684] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:32 2022] DELETE /identity/v3/users/7ae96cbf198740ec94313c7f0a5f6179 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:33.057353 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:33.079403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:33.088320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56d46d-cc63-4504-87fc-5bb2806422c1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:33.251090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bd917-5ba3-44a0-9756-059969a82234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:33.253092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bd917-5ba3-44a0-9756-059969a82234 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:33.255499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bd917-5ba3-44a0-9756-059969a82234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:33.256436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bd917-5ba3-44a0-9756-059969a82234 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:33.257579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bd917-5ba3-44a0-9756-059969a82234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:33.267635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bd917-5ba3-44a0-9756-059969a82234 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:33.332286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 842/1685] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:33 2022] POST /identity/v3/users => generated 294 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:33.336552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1eea0-494a-4960-93fc-2a225a5a666d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:33.338615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1eea0-494a-4960-93fc-2a225a5a666d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:33.338983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1eea0-494a-4960-93fc-2a225a5a666d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:33.367143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1ee9c-3c41-4b02-a5af-651ed737623d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:33.385287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d2d62-a037-4b12-b5f7-6bc4b22ba808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:33.387390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d2d62-a037-4b12-b5f7-6bc4b22ba808 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:33.389804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d2d62-a037-4b12-b5f7-6bc4b22ba808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:33.390320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d2d62-a037-4b12-b5f7-6bc4b22ba808 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:33.391252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d2d62-a037-4b12-b5f7-6bc4b22ba808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:33.400970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-afb1eea0-494a-4960-93fc-2a225a5a666d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:33.403195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d2d62-a037-4b12-b5f7-6bc4b22ba808 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:33.405498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-afb1eea0-494a-4960-93fc-2a225a5a666d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:33.427661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 843/1686] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:33.433418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1eea0-494a-4960-93fc-2a225a5a666d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:33.478834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1eea0-494a-4960-93fc-2a225a5a666d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:33.490206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 844/1687] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:33 2022] POST /identity/v3/projects => generated 381 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:33.493060 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dc738-ecd5-4e6a-92cd-7a07811ccd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:33.493808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dc738-ecd5-4e6a-92cd-7a07811ccd79 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:33.494065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dc738-ecd5-4e6a-92cd-7a07811ccd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:33.494492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d2d62-a037-4b12-b5f7-6bc4b22ba808 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:33.494967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dc738-ecd5-4e6a-92cd-7a07811ccd79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:33.496618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7dc738-ecd5-4e6a-92cd-7a07811ccd79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:33.500389 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fe0d2d62-a037-4b12-b5f7-6bc4b22ba808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:33.502091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7dc738-ecd5-4e6a-92cd-7a07811ccd79 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:33.572050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 844/1688] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:33.578873 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dc738-ecd5-4e6a-92cd-7a07811ccd79 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:33.623475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dc738-ecd5-4e6a-92cd-7a07811ccd79 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:33.827476 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d2d62-a037-4b12-b5f7-6bc4b22ba808 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:33.842733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402afd6c-6e67-4bc1-8ae6-ca0695dd2843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:33.858752 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402afd6c-6e67-4bc1-8ae6-ca0695dd2843 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:33.859096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402afd6c-6e67-4bc1-8ae6-ca0695dd2843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:33.859390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402afd6c-6e67-4bc1-8ae6-ca0695dd2843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:33.932085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7d2b4-1361-477e-ab40-60fe6e026c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:33.935713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7d2b4-1361-477e-ab40-60fe6e026c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:33.938640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7d2b4-1361-477e-ab40-60fe6e026c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:33.938640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7d2b4-1361-477e-ab40-60fe6e026c5d admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:33.942614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7d2b4-1361-477e-ab40-60fe6e026c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:33.953848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7d2b4-1361-477e-ab40-60fe6e026c5d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:34.022230 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 845/1689] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:33 2022] POST /identity/v3/users => generated 291 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:34.024957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7d2b4-1361-477e-ab40-60fe6e026c5d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:34.060812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7d2b4-1361-477e-ab40-60fe6e026c5d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:34.079555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285f6cf-8c1d-4737-8c8b-608ba4f47cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:34.081436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285f6cf-8c1d-4737-8c8b-608ba4f47cad admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:34.081724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285f6cf-8c1d-4737-8c8b-608ba4f47cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:34.082015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285f6cf-8c1d-4737-8c8b-608ba4f47cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:34.084096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285f6cf-8c1d-4737-8c8b-608ba4f47cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:34.091838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285f6cf-8c1d-4737-8c8b-608ba4f47cad admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:34.112986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402afd6c-6e67-4bc1-8ae6-ca0695dd2843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:34.116321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402afd6c-6e67-4bc1-8ae6-ca0695dd2843 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:34.127509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 846/1690] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:34.136156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285f6cf-8c1d-4737-8c8b-608ba4f47cad admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:34.140713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 845/1691] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:34.143382 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402afd6c-6e67-4bc1-8ae6-ca0695dd2843 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:34.160016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285f6cf-8c1d-4737-8c8b-608ba4f47cad admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:34.168023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402afd6c-6e67-4bc1-8ae6-ca0695dd2843 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:34.173025 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9830f3-5d58-4f5d-b9d9-30d9cbc4742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:34.173722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9830f3-5d58-4f5d-b9d9-30d9cbc4742c admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:34.175977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9830f3-5d58-4f5d-b9d9-30d9cbc4742c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:34.176264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9830f3-5d58-4f5d-b9d9-30d9cbc4742c admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:34.177086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9830f3-5d58-4f5d-b9d9-30d9cbc4742c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:34.179737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1db9e-8969-4996-aa14-40c2e422ee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:34.180685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9830f3-5d58-4f5d-b9d9-30d9cbc4742c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:34.180790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1db9e-8969-4996-aa14-40c2e422ee67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:34.181152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1db9e-8969-4996-aa14-40c2e422ee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:34.181431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1db9e-8969-4996-aa14-40c2e422ee67 admin admin] PATH_INFO: `/v3/projects/fde2705e191a41b1914b97cc8ef0b0c1/users/f00a8fe38a204a829b434fa09f544783/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:34.218658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 847/1692] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:34.219695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9830f3-5d58-4f5d-b9d9-30d9cbc4742c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:34.222888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1db9e-8969-4996-aa14-40c2e422ee67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde2705e191a41b1914b97cc8ef0b0c1, user_id=f00a8fe38a204a829b434fa09f544783, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:34.226366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1db9e-8969-4996-aa14-40c2e422ee67 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:34.235575 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:34.245594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9830f3-5d58-4f5d-b9d9-30d9cbc4742c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:34.253035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 846/1693] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:34 2022] PUT /identity/v3/projects/fde2705e191a41b1914b97cc8ef0b0c1/users/f00a8fe38a204a829b434fa09f544783/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:34.265270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff52dd2-1920-4496-b01d-0446332d740e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:34.265584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff52dd2-1920-4496-b01d-0446332d740e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:34.265827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff52dd2-1920-4496-b01d-0446332d740e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:34.276891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:34.327190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0ff52dd2-1920-4496-b01d-0446332d740e None None] MFA Rules not processed for user `f00a8fe38a204a829b434fa09f544783`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:34.331280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0ff52dd2-1920-4496-b01d-0446332d740e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:34.418225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786056de-9481-48ac-87df-c219e8a7ab2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:34.420248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786056de-9481-48ac-87df-c219e8a7ab2d admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:34.421260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786056de-9481-48ac-87df-c219e8a7ab2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:34.422250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786056de-9481-48ac-87df-c219e8a7ab2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:34.425375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786056de-9481-48ac-87df-c219e8a7ab2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:34.427555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786056de-9481-48ac-87df-c219e8a7ab2d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:34.458345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 848/1694] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:34.459813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786056de-9481-48ac-87df-c219e8a7ab2d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:34.461746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-786056de-9481-48ac-87df-c219e8a7ab2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:34.503335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786056de-9481-48ac-87df-c219e8a7ab2d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:34.516051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:34.518501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:34.518786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:34.519538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] PATH_INFO: `/v3/projects/988313c714f540718ce907c46ea7acde` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:34.522880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=988313c714f540718ce907c46ea7acde)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:34.541262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 847/1695] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:34.542962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff52dd2-1920-4496-b01d-0446332d740e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:34.543266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:34.577099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff52dd2-1920-4496-b01d-0446332d740e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:34.592891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271ffd9-133b-4649-a40d-fceb124ccb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:34.595393 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271ffd9-133b-4649-a40d-fceb124ccb77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:34.595680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271ffd9-133b-4649-a40d-fceb124ccb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:34.597537 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271ffd9-133b-4649-a40d-fceb124ccb77 admin admin] PATH_INFO: `/v3/projects/a88801a5139e40d0b4975a3213dfc8cf/users/2e7c8da56446429881ef241d2ce72538/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:34.684848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271ffd9-133b-4649-a40d-fceb124ccb77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a88801a5139e40d0b4975a3213dfc8cf, user_id=2e7c8da56446429881ef241d2ce72538, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:34.687723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271ffd9-133b-4649-a40d-fceb124ccb77 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:34.702574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '988313c714f540718ce907c46ea7acde', 'request_id': 'req-8ae1061b-0273-45b5-adcb-5cc85a00a905'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:34.726477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '988313c714f540718ce907c46ea7acde', 'request_id': 'req-8ae1061b-0273-45b5-adcb-5cc85a00a905'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:34.729297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 848/1696] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:34 2022] PUT /identity/v3/projects/a88801a5139e40d0b4975a3213dfc8cf/users/2e7c8da56446429881ef241d2ce72538/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:34.750026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271ffd9-133b-4649-a40d-fceb124ccb77 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:34.766273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '988313c714f540718ce907c46ea7acde', 'request_id': 'req-8ae1061b-0273-45b5-adcb-5cc85a00a905'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:34.813742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] The token cache is being invalidate because project 988313c714f540718ce907c46ea7acde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:34.815336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:34.817077 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8ae1061b-0273-45b5-adcb-5cc85a00a905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:34.829414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 849/1697] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:34 2022] DELETE /identity/v3/projects/988313c714f540718ce907c46ea7acde => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:34.830992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbf4df-339a-428f-a322-b83238d68105 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:34.831284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbf4df-339a-428f-a322-b83238d68105 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:34.831889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbf4df-339a-428f-a322-b83238d68105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:34.909933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-95dbf4df-339a-428f-a322-b83238d68105 None None] MFA Rules not processed for user `2e7c8da56446429881ef241d2ce72538`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:34.911455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-95dbf4df-339a-428f-a322-b83238d68105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:35.001647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271ffd9-133b-4649-a40d-fceb124ccb77 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:35.019338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da69d6-1a7c-49ac-a3ef-4fddc011cfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:35.020063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da69d6-1a7c-49ac-a3ef-4fddc011cfc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:35.020351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da69d6-1a7c-49ac-a3ef-4fddc011cfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:35.020668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da69d6-1a7c-49ac-a3ef-4fddc011cfc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:35.027616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da69d6-1a7c-49ac-a3ef-4fddc011cfc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:35.030072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da69d6-1a7c-49ac-a3ef-4fddc011cfc1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:35.097614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 849/1698] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:34 2022] POST /identity/v3/projects => generated 385 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:35.098561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da69d6-1a7c-49ac-a3ef-4fddc011cfc1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:35.103271 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c5da69d6-1a7c-49ac-a3ef-4fddc011cfc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:35.116695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 850/1699] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:35.118589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbf4df-339a-428f-a322-b83238d68105 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:35.122393 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-95dbf4df-339a-428f-a322-b83238d68105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:35.168762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbf4df-339a-428f-a322-b83238d68105 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:35.200527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:35.275800 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:35.312254 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da69d6-1a7c-49ac-a3ef-4fddc011cfc1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:35.336209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:35.337852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:35.339027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:35.339520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] PATH_INFO: `/v3/users/13730278f6e34516aa351a42e2e9a363` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:35.341762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47403ffb-a5fb-4424-a53c-56ae9edc2462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:35.342716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13730278f6e34516aa351a42e2e9a363)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:35.343330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403ffb-a5fb-4424-a53c-56ae9edc2462 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:35.344447 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403ffb-a5fb-4424-a53c-56ae9edc2462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:35.346387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:35.346826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403ffb-a5fb-4424-a53c-56ae9edc2462 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:35.347876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47403ffb-a5fb-4424-a53c-56ae9edc2462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:35.353361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47403ffb-a5fb-4424-a53c-56ae9edc2462 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:35.397939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 851/1700] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:35 2022] POST /identity/v3/users => generated 299 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:35.398994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47403ffb-a5fb-4424-a53c-56ae9edc2462 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:35.402650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-47403ffb-a5fb-4424-a53c-56ae9edc2462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:35.437321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47403ffb-a5fb-4424-a53c-56ae9edc2462 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:35.449939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b6b0a-05e8-493d-bc4f-51ae96e65505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:35.451730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6b0a-05e8-493d-bc4f-51ae96e65505 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:35.452936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6b0a-05e8-493d-bc4f-51ae96e65505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:35.454256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b6b0a-05e8-493d-bc4f-51ae96e65505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:35.455649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b6b0a-05e8-493d-bc4f-51ae96e65505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:35.462191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b6b0a-05e8-493d-bc4f-51ae96e65505 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:35.471082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13730278f6e34516aa351a42e2e9a363'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:35.473206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13730278f6e34516aa351a42e2e9a363'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:35.491169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13730278f6e34516aa351a42e2e9a363'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:35.493574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13730278f6e34516aa351a42e2e9a363'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:35.508306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a244c0b3-d65f-4ba2-81b0-a4efe8dd4430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13730278f6e34516aa351a42e2e9a363'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:35.568203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 852/1701] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:35 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:35.573129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b6b0a-05e8-493d-bc4f-51ae96e65505 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:35.575141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c83b6b0a-05e8-493d-bc4f-51ae96e65505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:35.583550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 850/1702] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:35 2022] DELETE /identity/v3/users/13730278f6e34516aa351a42e2e9a363 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:35.604715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2084e61-12d5-49e5-a4d6-580c46f51e37 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:35.606693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2084e61-12d5-49e5-a4d6-580c46f51e37 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:35.607005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2084e61-12d5-49e5-a4d6-580c46f51e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:35.718532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e2084e61-12d5-49e5-a4d6-580c46f51e37 None None] MFA Rules not processed for user `9e13bd970d2e4d67b62452ae0d6cae56`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:35.719895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e2084e61-12d5-49e5-a4d6-580c46f51e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:35.761708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b6b0a-05e8-493d-bc4f-51ae96e65505 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:35.772483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:35.775480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:35.775960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:35.777361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] PATH_INFO: `/v3/projects/8b762d2e87da463dac84eedfdedbeb7f` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:35.781043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b762d2e87da463dac84eedfdedbeb7f)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:35.788689 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:35.931453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b762d2e87da463dac84eedfdedbeb7f', 'request_id': 'req-5ec27105-1290-4c47-828d-fb05170b6dc6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:35.947326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b762d2e87da463dac84eedfdedbeb7f', 'request_id': 'req-5ec27105-1290-4c47-828d-fb05170b6dc6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:35.981321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b762d2e87da463dac84eedfdedbeb7f', 'request_id': 'req-5ec27105-1290-4c47-828d-fb05170b6dc6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:36.004208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] The token cache is being invalidate because project 8b762d2e87da463dac84eedfdedbeb7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:36.005127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:36.005680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:36.011801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 853/1703] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:35 2022] DELETE /identity/v3/projects/8b762d2e87da463dac84eedfdedbeb7f => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:36.012874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:36.019411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:36.055215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:36.075368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 851/1704] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:36.089678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ee3eb-32e8-471b-9231-8c8cffa75662 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:36.093694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ee3eb-32e8-471b-9231-8c8cffa75662 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:36.096029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ee3eb-32e8-471b-9231-8c8cffa75662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:36.157202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1e1ee3eb-32e8-471b-9231-8c8cffa75662 None None] MFA Rules not processed for user `d26f4426ba8c43399c4a7ef2f1701f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:36.159242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1e1ee3eb-32e8-471b-9231-8c8cffa75662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:36.162602 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec27105-1290-4c47-828d-fb05170b6dc6 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:36.176173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a26cb-986a-4a64-b3f0-7aeff15fed09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:36.176915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a26cb-986a-4a64-b3f0-7aeff15fed09 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:36.177194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a26cb-986a-4a64-b3f0-7aeff15fed09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:36.177477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a26cb-986a-4a64-b3f0-7aeff15fed09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:36.178380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a26cb-986a-4a64-b3f0-7aeff15fed09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:36.182028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a26cb-986a-4a64-b3f0-7aeff15fed09 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:36.208168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 854/1705] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:36.208168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a26cb-986a-4a64-b3f0-7aeff15fed09 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:36.208476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-640a26cb-986a-4a64-b3f0-7aeff15fed09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:36.236561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a26cb-986a-4a64-b3f0-7aeff15fed09 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:36.255567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600c40-ee9e-4db9-b9b1-86e8c15cb701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:36.256988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600c40-ee9e-4db9-b9b1-86e8c15cb701 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:36.257272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600c40-ee9e-4db9-b9b1-86e8c15cb701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:36.257547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600c40-ee9e-4db9-b9b1-86e8c15cb701 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:36.258273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18600c40-ee9e-4db9-b9b1-86e8c15cb701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:36.260301 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18600c40-ee9e-4db9-b9b1-86e8c15cb701 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:36.292202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 855/1706] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:36 2022] POST /identity/v3/users => generated 296 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:36.293184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600c40-ee9e-4db9-b9b1-86e8c15cb701 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:36.299903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-18600c40-ee9e-4db9-b9b1-86e8c15cb701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:36.344020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600c40-ee9e-4db9-b9b1-86e8c15cb701 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:36.348940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 852/1707] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Jun 8 02:05:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:36.350950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971a6ca-94e6-40a9-8715-da7dc685e3be None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:36.351288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971a6ca-94e6-40a9-8715-da7dc685e3be None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:36.351568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971a6ca-94e6-40a9-8715-da7dc685e3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:36.360493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:36.364175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:36.364326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:36.364690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] PATH_INFO: `/v3/users/86c9976371bf4594ae14b52d4cc921c9` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:36.373367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c9976371bf4594ae14b52d4cc921c9)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:36.378070 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:36.406826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c971a6ca-94e6-40a9-8715-da7dc685e3be None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:36.412203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c971a6ca-94e6-40a9-8715-da7dc685e3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:36.436375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 853/1708] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:36.438317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08574d-d2c5-4ea3-8985-5088f23dffc3 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:36.438612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08574d-d2c5-4ea3-8985-5088f23dffc3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:36.438892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08574d-d2c5-4ea3-8985-5088f23dffc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:36.450870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:36.477325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3f08574d-d2c5-4ea3-8985-5088f23dffc3 None None] MFA Rules not processed for user `9e13bd970d2e4d67b62452ae0d6cae56`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:36.477523 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c9976371bf4594ae14b52d4cc921c9'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:36.478153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c9976371bf4594ae14b52d4cc921c9'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:36.482933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3f08574d-d2c5-4ea3-8985-5088f23dffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:36.494725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c9976371bf4594ae14b52d4cc921c9'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:36.495707 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c9976371bf4594ae14b52d4cc921c9'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:36.546373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c9976371bf4594ae14b52d4cc921c9'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:36.586156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 856/1709] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:36 2022] DELETE /identity/v3/users/86c9976371bf4594ae14b52d4cc921c9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:36.587253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:36.589518 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:36.732699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 854/1710] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:36.733629 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08574d-d2c5-4ea3-8985-5088f23dffc3 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:36.744051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3f08574d-d2c5-4ea3-8985-5088f23dffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:36.772000 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d9899-4769-45ca-8c27-0ac4a0b19470 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:36.772138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08574d-d2c5-4ea3-8985-5088f23dffc3 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:36.783090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7741187-5938-4ce9-88ff-490076fab2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:36.783725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741187-5938-4ce9-88ff-490076fab2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:36.784001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741187-5938-4ce9-88ff-490076fab2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:36.784281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741187-5938-4ce9-88ff-490076fab2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:36.789380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7741187-5938-4ce9-88ff-490076fab2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:36.791350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7741187-5938-4ce9-88ff-490076fab2b3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:36.797279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b08f9-4184-4935-b7e0-b9bc894431d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:36.798648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b08f9-4184-4935-b7e0-b9bc894431d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:36.798935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b08f9-4184-4935-b7e0-b9bc894431d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:36.799219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b08f9-4184-4935-b7e0-b9bc894431d8 admin admin] PATH_INFO: `/v3/projects/77f347909002479584583958dc800b69/users/696147cd85ed4e0586d6b9673ef991a0/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:36.805800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:36.835354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 855/1711] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:36.836326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7741187-5938-4ce9-88ff-490076fab2b3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:36.842117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f7741187-5938-4ce9-88ff-490076fab2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:36.842311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b08f9-4184-4935-b7e0-b9bc894431d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f347909002479584583958dc800b69, user_id=696147cd85ed4e0586d6b9673ef991a0, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:36.852058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b08f9-4184-4935-b7e0-b9bc894431d8 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:36.878470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7741187-5938-4ce9-88ff-490076fab2b3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:36.879157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 857/1712] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:36 2022] PUT /identity/v3/projects/77f347909002479584583958dc800b69/users/696147cd85ed4e0586d6b9673ef991a0/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:36.880625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc082d-c673-4dd2-8748-a2782596383a None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:36.883729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc082d-c673-4dd2-8748-a2782596383a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:36.884075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc082d-c673-4dd2-8748-a2782596383a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:36.965675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-18fc082d-c673-4dd2-8748-a2782596383a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:36.969543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-18fc082d-c673-4dd2-8748-a2782596383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:37.069998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:37.071379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:37.071876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:37.072376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] PATH_INFO: `/v3/projects/6393d327003b4573bd16e88ea5b86aa5` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:37.073891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6393d327003b4573bd16e88ea5b86aa5)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:37.076862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:37.171048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6393d327003b4573bd16e88ea5b86aa5', 'request_id': 'req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:37.186185 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 858/1713] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:37.188102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6393d327003b4573bd16e88ea5b86aa5', 'request_id': 'req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:37.192094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc082d-c673-4dd2-8748-a2782596383a None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:37.195332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-18fc082d-c673-4dd2-8748-a2782596383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:37.218666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6393d327003b4573bd16e88ea5b86aa5', 'request_id': 'req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:37.268291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] The token cache is being invalidate because project 6393d327003b4573bd16e88ea5b86aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:37.268682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:37.269369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ca764c7c-fbc4-43b1-b4dc-e69960926d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:37.278370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 856/1714] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:36 2022] DELETE /identity/v3/projects/6393d327003b4573bd16e88ea5b86aa5 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:37.279938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afc617-3e6f-4661-a455-c896a4c33fe2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:37.280234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afc617-3e6f-4661-a455-c896a4c33fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:37.280551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afc617-3e6f-4661-a455-c896a4c33fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:37.321430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d7afc617-3e6f-4661-a455-c896a4c33fe2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:37.322723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d7afc617-3e6f-4661-a455-c896a4c33fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:37.379196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc082d-c673-4dd2-8748-a2782596383a None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:37.395662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:37.494129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 857/1715] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:37.498195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afc617-3e6f-4661-a455-c896a4c33fe2 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:37.499590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d7afc617-3e6f-4661-a455-c896a4c33fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:37.542938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2d952-240f-4f6c-b773-f8de67670f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:37.544828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d952-240f-4f6c-b773-f8de67670f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:37.545115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d952-240f-4f6c-b773-f8de67670f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:37.545492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d952-240f-4f6c-b773-f8de67670f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:37.557451 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-aac2d952-240f-4f6c-b773-f8de67670f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:37.573667 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2d952-240f-4f6c-b773-f8de67670f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:37.575158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2d952-240f-4f6c-b773-f8de67670f88 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:37.587499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afc617-3e6f-4661-a455-c896a4c33fe2 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:37.595251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e132b9-b280-4df3-90c9-dcd2aa3cd350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:37.595924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e132b9-b280-4df3-90c9-dcd2aa3cd350 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:37.596223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e132b9-b280-4df3-90c9-dcd2aa3cd350 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:37.596523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e132b9-b280-4df3-90c9-dcd2aa3cd350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:37.598509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f7e132b9-b280-4df3-90c9-dcd2aa3cd350 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:37.600526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 859/1716] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:37.602341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2d952-240f-4f6c-b773-f8de67670f88 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:37.605091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-aac2d952-240f-4f6c-b773-f8de67670f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:37.643059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2d952-240f-4f6c-b773-f8de67670f88 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:37.656609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9937968-71c3-4b13-bdaf-0452293e345d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:37.659692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9937968-71c3-4b13-bdaf-0452293e345d admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:37.660047 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9937968-71c3-4b13-bdaf-0452293e345d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:37.660583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9937968-71c3-4b13-bdaf-0452293e345d admin admin] PATH_INFO: `/v3/projects/6a590dc5d68d41f5982a10ca5a6eb748/users/43f0cdd162bb4c968314755c5b886985/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:37.702422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9937968-71c3-4b13-bdaf-0452293e345d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a590dc5d68d41f5982a10ca5a6eb748, user_id=43f0cdd162bb4c968314755c5b886985, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:37.711432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9937968-71c3-4b13-bdaf-0452293e345d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:37.732021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 860/1717] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:37 2022] PUT /identity/v3/projects/6a590dc5d68d41f5982a10ca5a6eb748/users/43f0cdd162bb4c968314755c5b886985/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:37.733169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9937968-71c3-4b13-bdaf-0452293e345d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:37.736339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b9937968-71c3-4b13-bdaf-0452293e345d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:37.743568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e132b9-b280-4df3-90c9-dcd2aa3cd350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:37.750469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e132b9-b280-4df3-90c9-dcd2aa3cd350 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:37.819645 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:37.846453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 858/1718] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:37.847958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e132b9-b280-4df3-90c9-dcd2aa3cd350 service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:37.848878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f7e132b9-b280-4df3-90c9-dcd2aa3cd350 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:37.916186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e132b9-b280-4df3-90c9-dcd2aa3cd350 service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:37.986287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9937968-71c3-4b13-bdaf-0452293e345d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:38.003405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526f58a-e9c4-4387-bbf1-9706b06482f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:38.004990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526f58a-e9c4-4387-bbf1-9706b06482f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.007884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526f58a-e9c4-4387-bbf1-9706b06482f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.009449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526f58a-e9c4-4387-bbf1-9706b06482f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.010261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526f58a-e9c4-4387-bbf1-9706b06482f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:38.013989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526f58a-e9c4-4387-bbf1-9706b06482f6 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:38.048326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 861/1719] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:38.049898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763ccb2-f0f0-45b1-8fd2-e6032d8c5c1b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.050195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763ccb2-f0f0-45b1-8fd2-e6032d8c5c1b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.050478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763ccb2-f0f0-45b1-8fd2-e6032d8c5c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.090643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-0763ccb2-f0f0-45b1-8fd2-e6032d8c5c1b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:38.091902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0763ccb2-f0f0-45b1-8fd2-e6032d8c5c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:38.095176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d58e-ca0a-44ae-a076-5961ba1d444f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:38.097442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d58e-ca0a-44ae-a076-5961ba1d444f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.097944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d58e-ca0a-44ae-a076-5961ba1d444f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.098938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d58e-ca0a-44ae-a076-5961ba1d444f admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.100996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847d58e-ca0a-44ae-a076-5961ba1d444f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:38.102062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847d58e-ca0a-44ae-a076-5961ba1d444f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:38.106226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847d58e-ca0a-44ae-a076-5961ba1d444f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:38.110865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 862/1720] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:38.111853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763ccb2-f0f0-45b1-8fd2-e6032d8c5c1b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:38.118847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0763ccb2-f0f0-45b1-8fd2-e6032d8c5c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:38.131623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:38.141877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 859/1721] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:38.141877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d58e-ca0a-44ae-a076-5961ba1d444f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:38.153444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d58e-ca0a-44ae-a076-5961ba1d444f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:38.161297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105a7fe-cbca-46dd-8ef1-4ebb88b73c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:38.162271 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105a7fe-cbca-46dd-8ef1-4ebb88b73c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.162625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105a7fe-cbca-46dd-8ef1-4ebb88b73c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.162983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105a7fe-cbca-46dd-8ef1-4ebb88b73c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.163846 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105a7fe-cbca-46dd-8ef1-4ebb88b73c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:38.165771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105a7fe-cbca-46dd-8ef1-4ebb88b73c3d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:38.190481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 860/1722] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:38.192257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105a7fe-cbca-46dd-8ef1-4ebb88b73c3d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:38.221820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105a7fe-cbca-46dd-8ef1-4ebb88b73c3d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:38.236241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae31ce-eff4-488e-94e6-4febf4c6f865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:38.237346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae31ce-eff4-488e-94e6-4febf4c6f865 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.237697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae31ce-eff4-488e-94e6-4febf4c6f865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.238051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae31ce-eff4-488e-94e6-4febf4c6f865 admin admin] PATH_INFO: `/v3/projects/77f347909002479584583958dc800b69/users/696147cd85ed4e0586d6b9673ef991a0/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.247165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae31ce-eff4-488e-94e6-4febf4c6f865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f347909002479584583958dc800b69, user_id=696147cd85ed4e0586d6b9673ef991a0, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:38.250974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae31ce-eff4-488e-94e6-4febf4c6f865 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:38.264282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763ccb2-f0f0-45b1-8fd2-e6032d8c5c1b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:38.268309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a752107-2c0b-4d6f-892b-4de8f99d6447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:38.269083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a752107-2c0b-4d6f-892b-4de8f99d6447 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.269444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a752107-2c0b-4d6f-892b-4de8f99d6447 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.269801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a752107-2c0b-4d6f-892b-4de8f99d6447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.279825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 861/1723] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:38 2022] PUT /identity/v3/projects/77f347909002479584583958dc800b69/users/696147cd85ed4e0586d6b9673ef991a0/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:38.281172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40aea4-b7ca-4125-8410-4457c3162e07 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.281476 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40aea4-b7ca-4125-8410-4457c3162e07 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.281813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40aea4-b7ca-4125-8410-4457c3162e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.299479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a752107-2c0b-4d6f-892b-4de8f99d6447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:38.302252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a752107-2c0b-4d6f-892b-4de8f99d6447 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:38.323206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-be40aea4-b7ca-4125-8410-4457c3162e07 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:38.324480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-be40aea4-b7ca-4125-8410-4457c3162e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:38.337480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 863/1724] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:38.339303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a752107-2c0b-4d6f-892b-4de8f99d6447 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:38.388101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a752107-2c0b-4d6f-892b-4de8f99d6447 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:38.598211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a937344-aa4b-4057-9765-bad15bf06d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:38.598888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a937344-aa4b-4057-9765-bad15bf06d56 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.599252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a937344-aa4b-4057-9765-bad15bf06d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.599806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a937344-aa4b-4057-9765-bad15bf06d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.600765 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a937344-aa4b-4057-9765-bad15bf06d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:38.609346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a937344-aa4b-4057-9765-bad15bf06d56 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:38.617348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 862/1725] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:38.619216 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40aea4-b7ca-4125-8410-4457c3162e07 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:38.651159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40aea4-b7ca-4125-8410-4457c3162e07 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:38.664111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b97a03-f21c-4cf3-b2f6-671101d9f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:38.665422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b97a03-f21c-4cf3-b2f6-671101d9f0ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.665500 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b97a03-f21c-4cf3-b2f6-671101d9f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.670956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b97a03-f21c-4cf3-b2f6-671101d9f0ea admin admin] PATH_INFO: `/v3/projects/6a590dc5d68d41f5982a10ca5a6eb748/users/43f0cdd162bb4c968314755c5b886985/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.676988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b97a03-f21c-4cf3-b2f6-671101d9f0ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a590dc5d68d41f5982a10ca5a6eb748, user_id=43f0cdd162bb4c968314755c5b886985, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:38.687560 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b97a03-f21c-4cf3-b2f6-671101d9f0ea admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:38.689434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 864/1726] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:38 2022] POST /identity/v3/projects => generated 405 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:38.692287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629459c3-7874-4519-9496-f45375b435d3 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.692631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629459c3-7874-4519-9496-f45375b435d3 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.696041 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629459c3-7874-4519-9496-f45375b435d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.738078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 863/1727] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:38 2022] PUT /identity/v3/projects/6a590dc5d68d41f5982a10ca5a6eb748/users/43f0cdd162bb4c968314755c5b886985/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:38.761343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b97a03-f21c-4cf3-b2f6-671101d9f0ea admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:38.809111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-629459c3-7874-4519-9496-f45375b435d3 None None] MFA Rules not processed for user `696147cd85ed4e0586d6b9673ef991a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:38.810458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-629459c3-7874-4519-9496-f45375b435d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:38.876878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:38.921575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b97a03-f21c-4cf3-b2f6-671101d9f0ea admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:38.944160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfe356-1a20-4a63-9ec7-b9edf0f5cc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:38.944160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfe356-1a20-4a63-9ec7-b9edf0f5cc8b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:38.944160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfe356-1a20-4a63-9ec7-b9edf0f5cc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:38.944592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfe356-1a20-4a63-9ec7-b9edf0f5cc8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:38.945010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfe356-1a20-4a63-9ec7-b9edf0f5cc8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:38.945317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfe356-1a20-4a63-9ec7-b9edf0f5cc8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:38.946961 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfe356-1a20-4a63-9ec7-b9edf0f5cc8b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:38.967554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 864/1728] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:38.968775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfe356-1a20-4a63-9ec7-b9edf0f5cc8b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:38.989479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfe356-1a20-4a63-9ec7-b9edf0f5cc8b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:39.001823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54769aea-a765-41ca-a770-2fbaaeefb01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:39.002940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54769aea-a765-41ca-a770-2fbaaeefb01d admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:39.003318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54769aea-a765-41ca-a770-2fbaaeefb01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:39.003807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54769aea-a765-41ca-a770-2fbaaeefb01d admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:39.005132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54769aea-a765-41ca-a770-2fbaaeefb01d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:39.006234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 865/1729] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:39.008477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54769aea-a765-41ca-a770-2fbaaeefb01d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:39.009407 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963c13b-e67f-446e-8d9b-99e8d371f11d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:39.010224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963c13b-e67f-446e-8d9b-99e8d371f11d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:39.010511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963c13b-e67f-446e-8d9b-99e8d371f11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:39.055099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-f963c13b-e67f-446e-8d9b-99e8d371f11d None None] MFA Rules not processed for user `43f0cdd162bb4c968314755c5b886985`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:39.057033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f963c13b-e67f-446e-8d9b-99e8d371f11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:39.062113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 865/1730] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:38 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:39.063418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54769aea-a765-41ca-a770-2fbaaeefb01d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:39.090435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54769aea-a765-41ca-a770-2fbaaeefb01d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:39.110231 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0f586-ac2e-4f5c-95de-acf525cfce6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:39.112474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0f586-ac2e-4f5c-95de-acf525cfce6b admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:39.112832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0f586-ac2e-4f5c-95de-acf525cfce6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:39.113194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0f586-ac2e-4f5c-95de-acf525cfce6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:39.114395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0f586-ac2e-4f5c-95de-acf525cfce6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:39.118636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0f586-ac2e-4f5c-95de-acf525cfce6b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:39.182038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 866/1731] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:39 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:39.183016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0f586-ac2e-4f5c-95de-acf525cfce6b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:39.192771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-33c0f586-ac2e-4f5c-95de-acf525cfce6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:39.254258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:39.273744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 866/1732] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:39.273744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963c13b-e67f-446e-8d9b-99e8d371f11d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:39.276639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f963c13b-e67f-446e-8d9b-99e8d371f11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:39.396104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0f586-ac2e-4f5c-95de-acf525cfce6b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:39.407651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963c13b-e67f-446e-8d9b-99e8d371f11d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:39.409179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:39.410381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:39.410593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:39.410875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] PATH_INFO: `/v3/projects/b6af942e37894522b760218345ebf027` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:39.413674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6af942e37894522b760218345ebf027)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:39.415640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:39.523922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820a60e-b7ac-4693-abc5-7660355b0c56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:39.524711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820a60e-b7ac-4693-abc5-7660355b0c56 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:39.525075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820a60e-b7ac-4693-abc5-7660355b0c56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:39.525565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820a60e-b7ac-4693-abc5-7660355b0c56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:39.557400 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:39.567783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6af942e37894522b760218345ebf027', 'request_id': 'req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:39.590856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6af942e37894522b760218345ebf027', 'request_id': 'req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:39.605928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6af942e37894522b760218345ebf027', 'request_id': 'req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:39.641661 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] The token cache is being invalidate because project b6af942e37894522b760218345ebf027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:39.641995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:39.643121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:39.651643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 867/1733] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:39 2022] DELETE /identity/v3/projects/b6af942e37894522b760218345ebf027 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:39.652760 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:39.656957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:39.684651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820a60e-b7ac-4693-abc5-7660355b0c56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:39.686798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820a60e-b7ac-4693-abc5-7660355b0c56 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:39.709594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 867/1734] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:39.710920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820a60e-b7ac-4693-abc5-7660355b0c56 service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:39.728085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820a60e-b7ac-4693-abc5-7660355b0c56 service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:39.850647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab18dd-7671-4291-a31a-f89b8cb92cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:39.851508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab18dd-7671-4291-a31a-f89b8cb92cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:39.852026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab18dd-7671-4291-a31a-f89b8cb92cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:39.852559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab18dd-7671-4291-a31a-f89b8cb92cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:39.853492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab18dd-7671-4291-a31a-f89b8cb92cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:39.860161 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab18dd-7671-4291-a31a-f89b8cb92cf5 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:39.930201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 868/1735] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:39 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:39.938228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab18dd-7671-4291-a31a-f89b8cb92cf5 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:39.945852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8aab18dd-7671-4291-a31a-f89b8cb92cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:40.010596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d94f3-dc48-4cb8-a931-e2b1f1ffc482 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:40.046952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8462c-53ef-4468-b435-166ae176ce8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:40.048055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8462c-53ef-4468-b435-166ae176ce8c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.048563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8462c-53ef-4468-b435-166ae176ce8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.049059 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8462c-53ef-4468-b435-166ae176ce8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.050137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8462c-53ef-4468-b435-166ae176ce8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:40.055697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8462c-53ef-4468-b435-166ae176ce8c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:40.090848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 868/1736] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:40.092669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0780865-ba89-4bed-8c78-b514d2efa337 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.097184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0780865-ba89-4bed-8c78-b514d2efa337 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.097712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0780865-ba89-4bed-8c78-b514d2efa337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.145906 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b0780865-ba89-4bed-8c78-b514d2efa337 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:40.147366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b0780865-ba89-4bed-8c78-b514d2efa337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:40.149691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab18dd-7671-4291-a31a-f89b8cb92cf5 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:40.158163 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aeb62-7e8b-4124-a2b2-e72b2948c30d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:40.159183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aeb62-7e8b-4124-a2b2-e72b2948c30d service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.161679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aeb62-7e8b-4124-a2b2-e72b2948c30d service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.161679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aeb62-7e8b-4124-a2b2-e72b2948c30d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.166129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-aa4aeb62-7e8b-4124-a2b2-e72b2948c30d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:40.175238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 869/1737] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:40.178103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0780865-ba89-4bed-8c78-b514d2efa337 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:40.180951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b0780865-ba89-4bed-8c78-b514d2efa337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:40.214551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0780865-ba89-4bed-8c78-b514d2efa337 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:40.228964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc7058-9660-4ff0-bed5-92abe3b6733d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:40.229656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc7058-9660-4ff0-bed5-92abe3b6733d admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.234589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc7058-9660-4ff0-bed5-92abe3b6733d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.234589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc7058-9660-4ff0-bed5-92abe3b6733d admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.238107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc7058-9660-4ff0-bed5-92abe3b6733d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:40.242728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc7058-9660-4ff0-bed5-92abe3b6733d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:40.278662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 870/1738] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:40.279619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc7058-9660-4ff0-bed5-92abe3b6733d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:40.299064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc7058-9660-4ff0-bed5-92abe3b6733d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:40.313425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6719cbc-aba4-4fa6-81aa-01af3e0688b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:40.313971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6719cbc-aba4-4fa6-81aa-01af3e0688b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.314325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6719cbc-aba4-4fa6-81aa-01af3e0688b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.314520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6719cbc-aba4-4fa6-81aa-01af3e0688b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.315248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6719cbc-aba4-4fa6-81aa-01af3e0688b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:40.316920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6719cbc-aba4-4fa6-81aa-01af3e0688b6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:40.326045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:40.340735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aeb62-7e8b-4124-a2b2-e72b2948c30d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:40.342321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aeb62-7e8b-4124-a2b2-e72b2948c30d service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:40.352487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 871/1739] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:40.354005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f15f2-c027-4a57-9018-4b451d891ef7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.354296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f15f2-c027-4a57-9018-4b451d891ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.356971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f15f2-c027-4a57-9018-4b451d891ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.366602 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 869/1740] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:40.367421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aeb62-7e8b-4124-a2b2-e72b2948c30d service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:40.393421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7c6f15f2-c027-4a57-9018-4b451d891ef7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:40.394643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7c6f15f2-c027-4a57-9018-4b451d891ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:40.396149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aeb62-7e8b-4124-a2b2-e72b2948c30d service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:40.414821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b9682-999f-45ad-a0f9-c6a51d99af06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:40.415570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b9682-999f-45ad-a0f9-c6a51d99af06 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.415789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b9682-999f-45ad-a0f9-c6a51d99af06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.416278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b9682-999f-45ad-a0f9-c6a51d99af06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.424013 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b9682-999f-45ad-a0f9-c6a51d99af06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:40.431749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b9682-999f-45ad-a0f9-c6a51d99af06 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:40.440118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 872/1741] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:40.441124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f15f2-c027-4a57-9018-4b451d891ef7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:40.480813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f15f2-c027-4a57-9018-4b451d891ef7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:40.481812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 870/1742] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:40.482764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b9682-999f-45ad-a0f9-c6a51d99af06 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:40.501901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5de0d-2eee-4da9-a1fd-251cf54c8a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:40.502815 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5de0d-2eee-4da9-a1fd-251cf54c8a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.503163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5de0d-2eee-4da9-a1fd-251cf54c8a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.503371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5de0d-2eee-4da9-a1fd-251cf54c8a7e admin admin] PATH_INFO: `/v3/projects/c75a902d9e5b4fdf9b423d579f900e02/users/ec488621fe5a4d31aa2be1204c239d82/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.508766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:40.512734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b9682-999f-45ad-a0f9-c6a51d99af06 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:40.524113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9beac55-308a-4f55-b937-ca005978ee0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:40.524864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beac55-308a-4f55-b937-ca005978ee0e admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.525256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beac55-308a-4f55-b937-ca005978ee0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.525441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9beac55-308a-4f55-b937-ca005978ee0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.526658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9beac55-308a-4f55-b937-ca005978ee0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:40.526865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9beac55-308a-4f55-b937-ca005978ee0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:40.528535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9beac55-308a-4f55-b937-ca005978ee0e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:40.544438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5de0d-2eee-4da9-a1fd-251cf54c8a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75a902d9e5b4fdf9b423d579f900e02, user_id=ec488621fe5a4d31aa2be1204c239d82, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:40.549629 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5de0d-2eee-4da9-a1fd-251cf54c8a7e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:40.584502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 871/1743] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:40.586481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9beac55-308a-4f55-b937-ca005978ee0e admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:40.588295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 873/1744] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:40 2022] PUT /identity/v3/projects/c75a902d9e5b4fdf9b423d579f900e02/users/ec488621fe5a4d31aa2be1204c239d82/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:40.593101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5de0d-2eee-4da9-a1fd-251cf54c8a7e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:40.616593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9beac55-308a-4f55-b937-ca005978ee0e admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:40.790669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83deaada-dba2-4dbb-aa03-f237c0c96422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:40.791534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83deaada-dba2-4dbb-aa03-f237c0c96422 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.791826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83deaada-dba2-4dbb-aa03-f237c0c96422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.792662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83deaada-dba2-4dbb-aa03-f237c0c96422 admin admin] PATH_INFO: `/v3/projects/657dc367a1104ad6a5570a446ae39985/users/6aa7db022b12421a956f9f0f5e52cbd9/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.811205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83deaada-dba2-4dbb-aa03-f237c0c96422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=657dc367a1104ad6a5570a446ae39985, user_id=6aa7db022b12421a956f9f0f5e52cbd9, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:40.814259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83deaada-dba2-4dbb-aa03-f237c0c96422 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:40.822621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5de0d-2eee-4da9-a1fd-251cf54c8a7e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:40.831393 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce96021-982f-4531-b751-d5f3ae1f111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:40.833220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce96021-982f-4531-b751-d5f3ae1f111a admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.835081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce96021-982f-4531-b751-d5f3ae1f111a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.835355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce96021-982f-4531-b751-d5f3ae1f111a admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.837191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce96021-982f-4531-b751-d5f3ae1f111a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:40.841931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce96021-982f-4531-b751-d5f3ae1f111a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:40.842249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 872/1745] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:40 2022] PUT /identity/v3/projects/657dc367a1104ad6a5570a446ae39985/users/6aa7db022b12421a956f9f0f5e52cbd9/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:40.854986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4025f3-059b-4729-860b-ca2a9b45ff8f None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:40.855292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4025f3-059b-4729-860b-ca2a9b45ff8f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:40.855568 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4025f3-059b-4729-860b-ca2a9b45ff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:40.889778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ed4025f3-059b-4729-860b-ca2a9b45ff8f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:40.894533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ed4025f3-059b-4729-860b-ca2a9b45ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:40.931044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 874/1746] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:40 2022] POST /identity/v3/projects => generated 373 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:40.931533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce96021-982f-4531-b751-d5f3ae1f111a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:41.056582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 873/1747] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:41.058280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76375df6-d1e7-4231-b212-89653017eaf7 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:41.058793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76375df6-d1e7-4231-b212-89653017eaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:41.059266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76375df6-d1e7-4231-b212-89653017eaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:41.065624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce96021-982f-4531-b751-d5f3ae1f111a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:41.076292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1c22f-925f-49ec-870f-e24900f2aa02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:41.077368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1c22f-925f-49ec-870f-e24900f2aa02 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:41.077654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1c22f-925f-49ec-870f-e24900f2aa02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:41.077929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1c22f-925f-49ec-870f-e24900f2aa02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:41.079817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cab1c22f-925f-49ec-870f-e24900f2aa02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:41.192504 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-76375df6-d1e7-4231-b212-89653017eaf7 None None] MFA Rules not processed for user `6aa7db022b12421a956f9f0f5e52cbd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:41.196350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-76375df6-d1e7-4231-b212-89653017eaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:41.218950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1c22f-925f-49ec-870f-e24900f2aa02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:41.220479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1c22f-925f-49ec-870f-e24900f2aa02 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:41.247811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 875/1748] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:41.258106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1c22f-925f-49ec-870f-e24900f2aa02 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:41.270170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:41.316707 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 874/1749] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:41.318518 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76375df6-d1e7-4231-b212-89653017eaf7 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:41.320702 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-76375df6-d1e7-4231-b212-89653017eaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:41.348061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76375df6-d1e7-4231-b212-89653017eaf7 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:41.353417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047324bf-fc3a-4460-a7f4-7c97192eb19e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:41.355273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047324bf-fc3a-4460-a7f4-7c97192eb19e service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:41.355793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047324bf-fc3a-4460-a7f4-7c97192eb19e service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:41.356317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047324bf-fc3a-4460-a7f4-7c97192eb19e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:41.358226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:41.361557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-047324bf-fc3a-4460-a7f4-7c97192eb19e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:41.471973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1c22f-925f-49ec-870f-e24900f2aa02 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:41.488831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e66d7-f39f-4c63-bcda-23b572280e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:41.489631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e66d7-f39f-4c63-bcda-23b572280e83 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:41.489947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e66d7-f39f-4c63-bcda-23b572280e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:41.490756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e66d7-f39f-4c63-bcda-23b572280e83 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:41.491575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e66d7-f39f-4c63-bcda-23b572280e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:41.501100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e66d7-f39f-4c63-bcda-23b572280e83 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:41.519423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047324bf-fc3a-4460-a7f4-7c97192eb19e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:41.524721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047324bf-fc3a-4460-a7f4-7c97192eb19e service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:41.545541 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 876/1750] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:41 2022] POST /identity/v3/users => generated 287 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:41.546403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e66d7-f39f-4c63-bcda-23b572280e83 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:41.550302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e48e66d7-f39f-4c63-bcda-23b572280e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:41.579515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 875/1751] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:41.580182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047324bf-fc3a-4460-a7f4-7c97192eb19e service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:41.637317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047324bf-fc3a-4460-a7f4-7c97192eb19e service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:41.651492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ced2ae-efe5-4f41-808b-0df7bb270f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:41.652249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ced2ae-efe5-4f41-808b-0df7bb270f7c admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:41.654475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ced2ae-efe5-4f41-808b-0df7bb270f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:41.654773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ced2ae-efe5-4f41-808b-0df7bb270f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:41.655495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ced2ae-efe5-4f41-808b-0df7bb270f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:41.663557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ced2ae-efe5-4f41-808b-0df7bb270f7c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:41.708192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e66d7-f39f-4c63-bcda-23b572280e83 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:41.715562 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cca3d-c451-4b24-a78b-44b62e0bc618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:41.716330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cca3d-c451-4b24-a78b-44b62e0bc618 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:41.717445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cca3d-c451-4b24-a78b-44b62e0bc618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:41.717754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cca3d-c451-4b24-a78b-44b62e0bc618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:41.743044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 876/1752] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:41 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:41.745022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ced2ae-efe5-4f41-808b-0df7bb270f7c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:41.753402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cca3d-c451-4b24-a78b-44b62e0bc618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:41.757672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cca3d-c451-4b24-a78b-44b62e0bc618 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:41.800451 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 877/1753] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:41.801413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cca3d-c451-4b24-a78b-44b62e0bc618 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:41.821463 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cca3d-c451-4b24-a78b-44b62e0bc618 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:41.970118 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ced2ae-efe5-4f41-808b-0df7bb270f7c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:41.979492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d2160-5d37-4c7f-b7f9-4d86d8bb79db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:41.980480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a6041-db02-4e25-b970-bd5b0dbc22f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:41.980630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d2160-5d37-4c7f-b7f9-4d86d8bb79db admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:41.984124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a6041-db02-4e25-b970-bd5b0dbc22f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:41.984648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a6041-db02-4e25-b970-bd5b0dbc22f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:41.985901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a6041-db02-4e25-b970-bd5b0dbc22f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:41.986770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a6041-db02-4e25-b970-bd5b0dbc22f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:41.991600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d2160-5d37-4c7f-b7f9-4d86d8bb79db admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:41.991600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d2160-5d37-4c7f-b7f9-4d86d8bb79db admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:41.994942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d2160-5d37-4c7f-b7f9-4d86d8bb79db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:41.997210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d2160-5d37-4c7f-b7f9-4d86d8bb79db admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:42.005065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a6041-db02-4e25-b970-bd5b0dbc22f3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:42.046518 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 877/1754] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:42.048981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d2160-5d37-4c7f-b7f9-4d86d8bb79db admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:42.097084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d2160-5d37-4c7f-b7f9-4d86d8bb79db admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:42.107799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 878/1755] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:41 2022] POST /identity/v3/users => generated 289 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:42.108686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a6041-db02-4e25-b970-bd5b0dbc22f3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:42.126721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc9f0-5911-472e-9394-08170976a048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:42.127436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc9f0-5911-472e-9394-08170976a048 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:42.132758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc9f0-5911-472e-9394-08170976a048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:42.133108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc9f0-5911-472e-9394-08170976a048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:42.133992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfc9f0-5911-472e-9394-08170976a048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:42.135891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfc9f0-5911-472e-9394-08170976a048 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:42.147295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a6041-db02-4e25-b970-bd5b0dbc22f3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:42.164469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0d0ee-585b-4117-89b7-5df82cd45325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:42.166054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d0ee-585b-4117-89b7-5df82cd45325 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:42.167880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d0ee-585b-4117-89b7-5df82cd45325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:42.168325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0d0ee-585b-4117-89b7-5df82cd45325 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:42.170676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0d0ee-585b-4117-89b7-5df82cd45325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:42.174556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0d0ee-585b-4117-89b7-5df82cd45325 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:42.198677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 878/1756] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:42.199818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc9f0-5911-472e-9394-08170976a048 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:42.220822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 879/1757] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:42.226878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0d0ee-585b-4117-89b7-5df82cd45325 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:42.244074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc9f0-5911-472e-9394-08170976a048 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:42.250706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0d0ee-585b-4117-89b7-5df82cd45325 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:42.261779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0208c-5d77-4ca9-81bc-0a7a19ec3fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:42.265914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0208c-5d77-4ca9-81bc-0a7a19ec3fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:42.267035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5641e-de06-48c8-a9d2-9f3c099e677a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:42.267533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0208c-5d77-4ca9-81bc-0a7a19ec3fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:42.267963 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0208c-5d77-4ca9-81bc-0a7a19ec3fb5 admin admin] PATH_INFO: `/v3/projects/d0a179fd2abc4e4eae4ecf26761efcc7/users/f3dbf37ddeee414684101d875e763a01/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:42.268076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5641e-de06-48c8-a9d2-9f3c099e677a admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:42.268337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5641e-de06-48c8-a9d2-9f3c099e677a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:42.268726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5641e-de06-48c8-a9d2-9f3c099e677a admin admin] PATH_INFO: `/v3/projects/1478a92784584041930bd14e6abab415/users/5c647ddeb9c642d7aef7bd1fc99cef3c/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:42.315709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0208c-5d77-4ca9-81bc-0a7a19ec3fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0a179fd2abc4e4eae4ecf26761efcc7, user_id=f3dbf37ddeee414684101d875e763a01, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:42.321224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0208c-5d77-4ca9-81bc-0a7a19ec3fb5 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:42.353623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5641e-de06-48c8-a9d2-9f3c099e677a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1478a92784584041930bd14e6abab415, user_id=5c647ddeb9c642d7aef7bd1fc99cef3c, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:42.363333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5641e-de06-48c8-a9d2-9f3c099e677a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:42.387138 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 879/1758] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:42 2022] PUT /identity/v3/projects/d0a179fd2abc4e4eae4ecf26761efcc7/users/f3dbf37ddeee414684101d875e763a01/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:42.390350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5506e-d368-425e-8d00-31716730eaaa None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:42.390350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5506e-d368-425e-8d00-31716730eaaa None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:42.390350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5506e-d368-425e-8d00-31716730eaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:42.420672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 880/1759] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:42 2022] PUT /identity/v3/projects/1478a92784584041930bd14e6abab415/users/5c647ddeb9c642d7aef7bd1fc99cef3c/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:42.425152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5641e-de06-48c8-a9d2-9f3c099e677a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:42.464961 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:42.586595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e6b5506e-d368-425e-8d00-31716730eaaa None None] MFA Rules not processed for user `f3dbf37ddeee414684101d875e763a01`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:42.587900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e6b5506e-d368-425e-8d00-31716730eaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:42.668898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:42.746600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5641e-de06-48c8-a9d2-9f3c099e677a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:42.784169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682bb70-ba63-4979-90f4-52e8a46c2cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:42.786829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682bb70-ba63-4979-90f4-52e8a46c2cce admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:42.787002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682bb70-ba63-4979-90f4-52e8a46c2cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:42.787378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682bb70-ba63-4979-90f4-52e8a46c2cce admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:42.790765 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682bb70-ba63-4979-90f4-52e8a46c2cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:42.792460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682bb70-ba63-4979-90f4-52e8a46c2cce admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:42.829499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 881/1760] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:42.829499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9516b75-716b-469c-b9ab-1b7ec072fdd1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:42.829499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9516b75-716b-469c-b9ab-1b7ec072fdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:42.829499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9516b75-716b-469c-b9ab-1b7ec072fdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:42.859997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b9516b75-716b-469c-b9ab-1b7ec072fdd1 None None] MFA Rules not processed for user `9e13bd970d2e4d67b62452ae0d6cae56`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:42.866829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b9516b75-716b-469c-b9ab-1b7ec072fdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:42.988732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 880/1761] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 601 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:42.989785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5506e-d368-425e-8d00-31716730eaaa None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:43.033740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5506e-d368-425e-8d00-31716730eaaa None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:43.050950 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f28db-8a0a-4174-840b-fd4c8254063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:43.051866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f28db-8a0a-4174-840b-fd4c8254063b admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:43.052144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f28db-8a0a-4174-840b-fd4c8254063b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:43.053096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f28db-8a0a-4174-840b-fd4c8254063b admin admin] PATH_INFO: `/v3/projects/1478a92784584041930bd14e6abab415/users/5c647ddeb9c642d7aef7bd1fc99cef3c/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:43.054207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 882/1762] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:43.055761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e70291-bf4c-4d4d-970a-aef91d695dff None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:43.056202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e70291-bf4c-4d4d-970a-aef91d695dff None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:43.056565 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e70291-bf4c-4d4d-970a-aef91d695dff None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:43.058405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 883/1763] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:43.059549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbfafa-e178-4020-ae43-b8e386f61d0c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:43.059840 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbfafa-e178-4020-ae43-b8e386f61d0c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:43.060121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbfafa-e178-4020-ae43-b8e386f61d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:43.069835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f28db-8a0a-4174-840b-fd4c8254063b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1478a92784584041930bd14e6abab415, user_id=5c647ddeb9c642d7aef7bd1fc99cef3c, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:43.072577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f28db-8a0a-4174-840b-fd4c8254063b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:43.091182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-16cbfafa-e178-4020-ae43-b8e386f61d0c None None] MFA Rules not processed for user `1b43d2b8b9084099b832332a565e741c`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:43.092923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-16cbfafa-e178-4020-ae43-b8e386f61d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:43.148384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 881/1764] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:42 2022] PUT /identity/v3/projects/1478a92784584041930bd14e6abab415/users/5c647ddeb9c642d7aef7bd1fc99cef3c/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:43.149418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f28db-8a0a-4174-840b-fd4c8254063b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:43.177593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-094f28db-8a0a-4174-840b-fd4c8254063b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:43.200965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 884/1765] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:43.202315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cc45e-aa0b-45c8-b629-3d317bf62b85 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:43.202627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cc45e-aa0b-45c8-b629-3d317bf62b85 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:43.202882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cc45e-aa0b-45c8-b629-3d317bf62b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:43.337330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-af0cc45e-aa0b-45c8-b629-3d317bf62b85 None None] MFA Rules not processed for user `5c647ddeb9c642d7aef7bd1fc99cef3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:43.345704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-af0cc45e-aa0b-45c8-b629-3d317bf62b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:43.470714 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f28db-8a0a-4174-840b-fd4c8254063b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:43.480752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4f79f-dbb5-4a53-bf8a-ddcac039bbbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:43.481567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4f79f-dbb5-4a53-bf8a-ddcac039bbbe service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:43.481824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4f79f-dbb5-4a53-bf8a-ddcac039bbbe service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:43.482789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4f79f-dbb5-4a53-bf8a-ddcac039bbbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:43.565703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:43.639594 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 885/1766] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:43 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:43.640961 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936367e-42f1-4398-b2b8-15aae7a055d9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:43.641268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936367e-42f1-4398-b2b8-15aae7a055d9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:43.641915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936367e-42f1-4398-b2b8-15aae7a055d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:43.690709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c936367e-42f1-4398-b2b8-15aae7a055d9 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:43.691971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c936367e-42f1-4398-b2b8-15aae7a055d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:43.721002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 886/1767] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:05:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:43.722394 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936367e-42f1-4398-b2b8-15aae7a055d9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:43.734993 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4f79f-dbb5-4a53-bf8a-ddcac039bbbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:43.742010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4f79f-dbb5-4a53-bf8a-ddcac039bbbe service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:43.835976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 882/1768] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:44.024511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936367e-42f1-4398-b2b8-15aae7a055d9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:44.039475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f323e-0cb5-4a21-bb8e-4e736434980b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:44.040220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f323e-0cb5-4a21-bb8e-4e736434980b service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:44.043742 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f323e-0cb5-4a21-bb8e-4e736434980b service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:44.044075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f323e-0cb5-4a21-bb8e-4e736434980b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:44.086997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f323e-0cb5-4a21-bb8e-4e736434980b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:44.091003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f323e-0cb5-4a21-bb8e-4e736434980b service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:44.132431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 887/1769] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:44.256406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4f79f-dbb5-4a53-bf8a-ddcac039bbbe service nova] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:44.271015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4f79f-dbb5-4a53-bf8a-ddcac039bbbe service nova] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:44.280824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effdd86-1581-44ff-b816-0923312c535b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:44.282154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effdd86-1581-44ff-b816-0923312c535b service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:44.282838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effdd86-1581-44ff-b816-0923312c535b service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:44.283413 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effdd86-1581-44ff-b816-0923312c535b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:44.370612 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:44.435802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effdd86-1581-44ff-b816-0923312c535b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:44.437438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effdd86-1581-44ff-b816-0923312c535b service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:44.471940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 883/1770] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:44.837878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e292f-aed7-4cbd-ad52-aecd39b791da None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:44.838516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e292f-aed7-4cbd-ad52-aecd39b791da None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:44.838516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e292f-aed7-4cbd-ad52-aecd39b791da None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:44.840700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 888/1771] 10.222.0.20 () {54 vars in 912 bytes} [Wed Jun 8 02:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:44.846570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c963f-3d12-4379-86a3-b211a94f7ce6 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:44.847064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c963f-3d12-4379-86a3-b211a94f7ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:44.847508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c963f-3d12-4379-86a3-b211a94f7ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:44.875717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-893c963f-3d12-4379-86a3-b211a94f7ce6 None None] MFA Rules not processed for user `d26f4426ba8c43399c4a7ef2f1701f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:44.877595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-893c963f-3d12-4379-86a3-b211a94f7ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:45.027372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 884/1772] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:05:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:45.123416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e292f-aed7-4cbd-ad52-aecd39b791da None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:45.126258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-012e292f-aed7-4cbd-ad52-aecd39b791da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:45.167345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e292f-aed7-4cbd-ad52-aecd39b791da None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:45.171377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3a91c-4cb1-403d-b38e-de2d099bef4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:45.172047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3a91c-4cb1-403d-b38e-de2d099bef4a service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:45.172336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3a91c-4cb1-403d-b38e-de2d099bef4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:45.172659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3a91c-4cb1-403d-b38e-de2d099bef4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:45.188340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3a91c-4cb1-403d-b38e-de2d099bef4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:45.190557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3a91c-4cb1-403d-b38e-de2d099bef4a service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:45.225341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 889/1773] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:45.590382 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808779e-fb18-41cf-b64a-1602c6f6bfce None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:45.591567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808779e-fb18-41cf-b64a-1602c6f6bfce None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:45.592127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808779e-fb18-41cf-b64a-1602c6f6bfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:45.670159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-4808779e-fb18-41cf-b64a-1602c6f6bfce None None] MFA Rules not processed for user `ec488621fe5a4d31aa2be1204c239d82`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:45.671069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4808779e-fb18-41cf-b64a-1602c6f6bfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:45.805853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 885/1774] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:45 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:45.809910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3a91c-4cb1-403d-b38e-de2d099bef4a service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:45.899283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3a91c-4cb1-403d-b38e-de2d099bef4a service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:45.905929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b811b-1d50-4343-8acc-bfe3223abf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:45.907154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b811b-1d50-4343-8acc-bfe3223abf8f admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:45.913078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b811b-1d50-4343-8acc-bfe3223abf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:45.913804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b811b-1d50-4343-8acc-bfe3223abf8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:45.915400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b811b-1d50-4343-8acc-bfe3223abf8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:45.920270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b811b-1d50-4343-8acc-bfe3223abf8f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:45.951539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 890/1775] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:45 2022] POST /identity/v3/projects => generated 405 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:05:45.958730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808779e-fb18-41cf-b64a-1602c6f6bfce None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:45.973566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808779e-fb18-41cf-b64a-1602c6f6bfce None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:46.053253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7842398-6c2a-484c-9ddc-9984fbd35bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:46.054407 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7842398-6c2a-484c-9ddc-9984fbd35bfc admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:46.054900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7842398-6c2a-484c-9ddc-9984fbd35bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:46.055375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7842398-6c2a-484c-9ddc-9984fbd35bfc admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:46.056483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7842398-6c2a-484c-9ddc-9984fbd35bfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:46.058902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7842398-6c2a-484c-9ddc-9984fbd35bfc admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:46.077790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:46.087465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 886/1776] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:45 2022] POST /identity/v3/users => generated 309 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:05:46.091297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b811b-1d50-4343-8acc-bfe3223abf8f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:46.102587 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b811b-1d50-4343-8acc-bfe3223abf8f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:46.120936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b90b-e9fc-4b90-9e1b-48475971f405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:46.122065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b90b-e9fc-4b90-9e1b-48475971f405 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:46.122633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b90b-e9fc-4b90-9e1b-48475971f405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:46.123436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b90b-e9fc-4b90-9e1b-48475971f405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:46.125739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826b90b-e9fc-4b90-9e1b-48475971f405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:46.128199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826b90b-e9fc-4b90-9e1b-48475971f405 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:46.159577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 891/1777] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:46.160209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7842398-6c2a-484c-9ddc-9984fbd35bfc admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:46.177114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7842398-6c2a-484c-9ddc-9984fbd35bfc admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:46.181376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b90b-e9fc-4b90-9e1b-48475971f405 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:46.190058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c90ebb-ca01-4f9a-831d-7f39096aca0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:46.190977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c90ebb-ca01-4f9a-831d-7f39096aca0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:46.191262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c90ebb-ca01-4f9a-831d-7f39096aca0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:46.191534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c90ebb-ca01-4f9a-831d-7f39096aca0a admin admin] PATH_INFO: `/v3/projects/9043fd9359424b3aba901331410be340/users/e406d8eae1c6468a89a90d86fb672441/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:46.225945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c90ebb-ca01-4f9a-831d-7f39096aca0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9043fd9359424b3aba901331410be340, user_id=e406d8eae1c6468a89a90d86fb672441, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:46.230488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c90ebb-ca01-4f9a-831d-7f39096aca0a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:46.258856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 887/1778] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:46 2022] PUT /identity/v3/projects/9043fd9359424b3aba901331410be340/users/e406d8eae1c6468a89a90d86fb672441/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:46.263071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c90ebb-ca01-4f9a-831d-7f39096aca0a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:46.265832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:46.323490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b90b-e9fc-4b90-9e1b-48475971f405 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:46.388537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c90ebb-ca01-4f9a-831d-7f39096aca0a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:46.400630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af5c32-4d66-4837-aca6-0c11552383ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:46.403626 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af5c32-4d66-4837-aca6-0c11552383ed admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:46.403921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af5c32-4d66-4837-aca6-0c11552383ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:46.404988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af5c32-4d66-4837-aca6-0c11552383ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:46.406692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af5c32-4d66-4837-aca6-0c11552383ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:46.409386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af5c32-4d66-4837-aca6-0c11552383ed admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:46.442241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 888/1779] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:46.443342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af5c32-4d66-4837-aca6-0c11552383ed admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:46.454331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af5c32-4d66-4837-aca6-0c11552383ed admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:46.460631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452558ba-8d22-4341-8de6-e08b2e2a570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:46.461760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452558ba-8d22-4341-8de6-e08b2e2a570c admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:46.462275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452558ba-8d22-4341-8de6-e08b2e2a570c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:46.462729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452558ba-8d22-4341-8de6-e08b2e2a570c admin admin] PATH_INFO: `/v3/projects/9043fd9359424b3aba901331410be340/users/e406d8eae1c6468a89a90d86fb672441/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:46.465334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452558ba-8d22-4341-8de6-e08b2e2a570c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9043fd9359424b3aba901331410be340, user_id=e406d8eae1c6468a89a90d86fb672441, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:46.468709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452558ba-8d22-4341-8de6-e08b2e2a570c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:46.492473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecd44b-9d61-4aa7-800e-7a17cecf3d86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:46.493192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecd44b-9d61-4aa7-800e-7a17cecf3d86 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:46.493321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 889/1780] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:46 2022] PUT /identity/v3/projects/9043fd9359424b3aba901331410be340/users/e406d8eae1c6468a89a90d86fb672441/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:46.493927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecd44b-9d61-4aa7-800e-7a17cecf3d86 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:46.494359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecd44b-9d61-4aa7-800e-7a17cecf3d86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:46.507669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ecd44b-9d61-4aa7-800e-7a17cecf3d86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:46.509446 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ecd44b-9d61-4aa7-800e-7a17cecf3d86 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:46.541700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 892/1781] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:05:46.982744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452558ba-8d22-4341-8de6-e08b2e2a570c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:47.055552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452558ba-8d22-4341-8de6-e08b2e2a570c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:47.060286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:47.061217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:47.061503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:47.061782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] PATH_INFO: `/v3/users/f3dbf37ddeee414684101d875e763a01` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:47.064449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3dbf37ddeee414684101d875e763a01)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:47.066580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:47.117149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3dbf37ddeee414684101d875e763a01'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:47.118141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3dbf37ddeee414684101d875e763a01'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:47.125524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3dbf37ddeee414684101d875e763a01'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:47.126220 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3dbf37ddeee414684101d875e763a01'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:47.132281 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:47.143192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3dbf37ddeee414684101d875e763a01'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:47.159502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 890/1782] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:46 2022] DELETE /identity/v3/users/f3dbf37ddeee414684101d875e763a01 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:47.164245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e28dd-0152-496b-8a7d-153f44d80020 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:47.164777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e28dd-0152-496b-8a7d-153f44d80020 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:47.164883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e28dd-0152-496b-8a7d-153f44d80020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:47.184290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ad8e28dd-0152-496b-8a7d-153f44d80020 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:47.185642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ad8e28dd-0152-496b-8a7d-153f44d80020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:47.253466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 893/1783] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:47.263621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:47.265137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:47.326304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c637a-240a-4d82-b6bd-8389b1fb7bc1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:47.330506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504593b-6aa1-4192-8edf-56bbbe567810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:47.331153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504593b-6aa1-4192-8edf-56bbbe567810 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:47.331436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504593b-6aa1-4192-8edf-56bbbe567810 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:47.331709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504593b-6aa1-4192-8edf-56bbbe567810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:47.345596 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504593b-6aa1-4192-8edf-56bbbe567810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:47.347139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504593b-6aa1-4192-8edf-56bbbe567810 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:47.357900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 891/1784] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:47.429281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ddc17-1d1c-4426-9039-0eb03a116273 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:47.429798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ddc17-1d1c-4426-9039-0eb03a116273 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:47.430374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ddc17-1d1c-4426-9039-0eb03a116273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:47.480241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-952ddc17-1d1c-4426-9039-0eb03a116273 None None] MFA Rules not processed for user `e406d8eae1c6468a89a90d86fb672441`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:47.482780 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-952ddc17-1d1c-4426-9039-0eb03a116273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:47.541971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:47.571099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 894/1785] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:47 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:47.786910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504593b-6aa1-4192-8edf-56bbbe567810 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:47.788332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5504593b-6aa1-4192-8edf-56bbbe567810 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:47.832923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504593b-6aa1-4192-8edf-56bbbe567810 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:47.850912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:47.851805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:47.852094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:47.852373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] PATH_INFO: `/v3/projects/d0a179fd2abc4e4eae4ecf26761efcc7` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:47.855237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0a179fd2abc4e4eae4ecf26761efcc7)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:47.857212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:47.930783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0a179fd2abc4e4eae4ecf26761efcc7', 'request_id': 'req-b5827302-2a6d-4026-aaa8-0e75dd7bac92'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:47.940251 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0a179fd2abc4e4eae4ecf26761efcc7', 'request_id': 'req-b5827302-2a6d-4026-aaa8-0e75dd7bac92'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:47.962225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0a179fd2abc4e4eae4ecf26761efcc7', 'request_id': 'req-b5827302-2a6d-4026-aaa8-0e75dd7bac92'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:47.988228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] The token cache is being invalidate because project d0a179fd2abc4e4eae4ecf26761efcc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:47.988861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:47.989404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-b5827302-2a6d-4026-aaa8-0e75dd7bac92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:47.991392 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 892/1786] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:47 2022] DELETE /identity/v3/projects/d0a179fd2abc4e4eae4ecf26761efcc7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:47.996256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0c850-bf2d-421a-9acd-51620231c15c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:47.996677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0c850-bf2d-421a-9acd-51620231c15c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:47.997029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0c850-bf2d-421a-9acd-51620231c15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.015930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-07f0c850-bf2d-421a-9acd-51620231c15c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:48.017391 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-07f0c850-bf2d-421a-9acd-51620231c15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:48.085066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 895/1787] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:48.088544 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8486-a46f-4428-81b2-d59be318c66c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.088986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8486-a46f-4428-81b2-d59be318c66c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.089302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8486-a46f-4428-81b2-d59be318c66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.108060 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-bc1d8486-a46f-4428-81b2-d59be318c66c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:48.109395 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-bc1d8486-a46f-4428-81b2-d59be318c66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:48.121222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 893/1788] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:48.122653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0c850-bf2d-421a-9acd-51620231c15c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.132693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0c850-bf2d-421a-9acd-51620231c15c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.137401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805e4ce-8df0-4257-b360-b9be6d7d1b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.138079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805e4ce-8df0-4257-b360-b9be6d7d1b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.138348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805e4ce-8df0-4257-b360-b9be6d7d1b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.138623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805e4ce-8df0-4257-b360-b9be6d7d1b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.139347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805e4ce-8df0-4257-b360-b9be6d7d1b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:48.139655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805e4ce-8df0-4257-b360-b9be6d7d1b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:48.141314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805e4ce-8df0-4257-b360-b9be6d7d1b14 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:48.153239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 896/1789] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:48.157471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d8486-a46f-4428-81b2-d59be318c66c None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.167905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d8486-a46f-4428-81b2-d59be318c66c None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.172602 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e437ae9-6ad1-4e9a-8ea5-2f710dcfcefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.173255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e437ae9-6ad1-4e9a-8ea5-2f710dcfcefb admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.173542 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e437ae9-6ad1-4e9a-8ea5-2f710dcfcefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.173819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e437ae9-6ad1-4e9a-8ea5-2f710dcfcefb admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.174548 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e437ae9-6ad1-4e9a-8ea5-2f710dcfcefb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:48.176480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e437ae9-6ad1-4e9a-8ea5-2f710dcfcefb admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:48.205346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 894/1790] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:48 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:48.206428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805e4ce-8df0-4257-b360-b9be6d7d1b14 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.216545 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805e4ce-8df0-4257-b360-b9be6d7d1b14 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.277858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951e0e4-5792-4cff-ba23-20e508016160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.278657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951e0e4-5792-4cff-ba23-20e508016160 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.278947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951e0e4-5792-4cff-ba23-20e508016160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.279227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951e0e4-5792-4cff-ba23-20e508016160 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.279936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951e0e4-5792-4cff-ba23-20e508016160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:48.282032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951e0e4-5792-4cff-ba23-20e508016160 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:48.307030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 897/1791] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:48 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:05:48.308406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e437ae9-6ad1-4e9a-8ea5-2f710dcfcefb admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.319569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e437ae9-6ad1-4e9a-8ea5-2f710dcfcefb admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.324343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d595738-8e1e-416e-a012-4bfc8327be5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.325061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d595738-8e1e-416e-a012-4bfc8327be5f admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.325346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d595738-8e1e-416e-a012-4bfc8327be5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.325622 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d595738-8e1e-416e-a012-4bfc8327be5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.326372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d595738-8e1e-416e-a012-4bfc8327be5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:48.328004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d595738-8e1e-416e-a012-4bfc8327be5f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:48.341226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 895/1792] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:48.345098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951e0e4-5792-4cff-ba23-20e508016160 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.356218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951e0e4-5792-4cff-ba23-20e508016160 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.361005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303193a-054d-4b5c-aeb0-11f30a339b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.361687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303193a-054d-4b5c-aeb0-11f30a339b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.361976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303193a-054d-4b5c-aeb0-11f30a339b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.362253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303193a-054d-4b5c-aeb0-11f30a339b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.363022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303193a-054d-4b5c-aeb0-11f30a339b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:48.365220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303193a-054d-4b5c-aeb0-11f30a339b0b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:48.379210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 898/1793] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:05:48.380567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d595738-8e1e-416e-a012-4bfc8327be5f admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.391481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d595738-8e1e-416e-a012-4bfc8327be5f admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.396971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dec78d-dd2b-425f-ac6c-60621cd8632b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.397885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dec78d-dd2b-425f-ac6c-60621cd8632b admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.398170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dec78d-dd2b-425f-ac6c-60621cd8632b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.398449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dec78d-dd2b-425f-ac6c-60621cd8632b admin admin] PATH_INFO: `/v3/projects/535b95f50fc4478da650335cd45e6caf/users/fcf9a36ca46b444d88a195b9698290eb/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.414207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dec78d-dd2b-425f-ac6c-60621cd8632b admin admin] RBAC: Authorizing `identity:create_grant(project_id=535b95f50fc4478da650335cd45e6caf, user_id=fcf9a36ca46b444d88a195b9698290eb, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:48.416997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dec78d-dd2b-425f-ac6c-60621cd8632b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:48.426928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:48.433137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 896/1794] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:48 2022] PUT /identity/v3/projects/535b95f50fc4478da650335cd45e6caf/users/fcf9a36ca46b444d88a195b9698290eb/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:48.443679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867b15d-1d98-4f85-ac9c-9e45603d6a15 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.444177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867b15d-1d98-4f85-ac9c-9e45603d6a15 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.444390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867b15d-1d98-4f85-ac9c-9e45603d6a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.469147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f094ba-d71b-4e81-a2e4-6e97e05baaa3 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.469913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f094ba-d71b-4e81-a2e4-6e97e05baaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.470272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f094ba-d71b-4e81-a2e4-6e97e05baaa3 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.471753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 897/1795] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:48.472858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1867b15d-1d98-4f85-ac9c-9e45603d6a15 None None] MFA Rules not processed for user `fcf9a36ca46b444d88a195b9698290eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:48.474153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1867b15d-1d98-4f85-ac9c-9e45603d6a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:48.508529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30656a-7980-45af-8d66-6a2bce06b385 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.508904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30656a-7980-45af-8d66-6a2bce06b385 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.509201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30656a-7980-45af-8d66-6a2bce06b385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.530349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:48.536674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-de30656a-7980-45af-8d66-6a2bce06b385 None None] MFA Rules not processed for user `4eb69af145154c6d85bd4b845979e6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:48.538910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-de30656a-7980-45af-8d66-6a2bce06b385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:48.567135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 899/1796] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:48.571782 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867b15d-1d98-4f85-ac9c-9e45603d6a15 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.621823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 898/1797] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:05:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:48.635370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30656a-7980-45af-8d66-6a2bce06b385 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.637186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-de30656a-7980-45af-8d66-6a2bce06b385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:48.650749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30656a-7980-45af-8d66-6a2bce06b385 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.650897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867b15d-1d98-4f85-ac9c-9e45603d6a15 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.656030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a757-d4e9-444b-8fbb-4a71d83f1be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.660344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9edb1-9008-433b-925a-e005406d34ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.660344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9edb1-9008-433b-925a-e005406d34ba service placement] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.663056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a757-d4e9-444b-8fbb-4a71d83f1be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.663701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9edb1-9008-433b-925a-e005406d34ba service placement] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.665762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a757-d4e9-444b-8fbb-4a71d83f1be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.665875 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9edb1-9008-433b-925a-e005406d34ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.665966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a757-d4e9-444b-8fbb-4a71d83f1be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.665966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944a757-d4e9-444b-8fbb-4a71d83f1be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:48.665966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944a757-d4e9-444b-8fbb-4a71d83f1be7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:48.668687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9edb1-9008-433b-925a-e005406d34ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:48.670912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9edb1-9008-433b-925a-e005406d34ba service placement] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:48.686276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 899/1798] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:05:48.689973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 900/1799] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:48 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:48.697937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9edb1-9008-433b-925a-e005406d34ba service placement] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.711667 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9edb1-9008-433b-925a-e005406d34ba service placement] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.805715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f4937-a9ea-4b46-a30e-23ec73441728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.806398 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f4937-a9ea-4b46-a30e-23ec73441728 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.806686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f4937-a9ea-4b46-a30e-23ec73441728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.807660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f4937-a9ea-4b46-a30e-23ec73441728 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.809820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f4937-a9ea-4b46-a30e-23ec73441728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:48.810538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f4937-a9ea-4b46-a30e-23ec73441728 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:48.845975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 900/1800] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:48 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:48.850378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a757-d4e9-444b-8fbb-4a71d83f1be7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.873420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a757-d4e9-444b-8fbb-4a71d83f1be7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.886070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774079cb-9116-4af8-987e-d7772756a3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.887274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774079cb-9116-4af8-987e-d7772756a3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.887632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774079cb-9116-4af8-987e-d7772756a3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.888024 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774079cb-9116-4af8-987e-d7772756a3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:48.889132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774079cb-9116-4af8-987e-d7772756a3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:48.891117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774079cb-9116-4af8-987e-d7772756a3bd admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:48.907535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 901/1801] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:48.910528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f4937-a9ea-4b46-a30e-23ec73441728 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:48.930543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f4937-a9ea-4b46-a30e-23ec73441728 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:48.943264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769942c-e96a-4882-816f-13a10346886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:48.945593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769942c-e96a-4882-816f-13a10346886f admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:48.947132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769942c-e96a-4882-816f-13a10346886f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:48.948112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769942c-e96a-4882-816f-13a10346886f admin admin] PATH_INFO: `/v3/projects/98cd7da92abf4da3a76b4b278c7dc597/users/976e0488f3e7473e848ab529ea8362b2/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:49.016340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769942c-e96a-4882-816f-13a10346886f admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cd7da92abf4da3a76b4b278c7dc597, user_id=976e0488f3e7473e848ab529ea8362b2, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:49.019480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769942c-e96a-4882-816f-13a10346886f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:49.063012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 901/1802] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:48 2022] PUT /identity/v3/projects/98cd7da92abf4da3a76b4b278c7dc597/users/976e0488f3e7473e848ab529ea8362b2/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:49.067219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774079cb-9116-4af8-987e-d7772756a3bd admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:49.092323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774079cb-9116-4af8-987e-d7772756a3bd admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:49.187123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a08d1-a2e9-43fc-9e91-b1be6ee6a932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:49.188725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a08d1-a2e9-43fc-9e91-b1be6ee6a932 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:49.189367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a08d1-a2e9-43fc-9e91-b1be6ee6a932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:49.189642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a08d1-a2e9-43fc-9e91-b1be6ee6a932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:49.190526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a08d1-a2e9-43fc-9e91-b1be6ee6a932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:49.192173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a08d1-a2e9-43fc-9e91-b1be6ee6a932 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:49.214114 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 902/1803] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:49.215912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769942c-e96a-4882-816f-13a10346886f admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:49.229864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769942c-e96a-4882-816f-13a10346886f admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:49.235393 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822092c-5d2b-45fc-87dd-d1ffcef765ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:49.236736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822092c-5d2b-45fc-87dd-d1ffcef765ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:49.237280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822092c-5d2b-45fc-87dd-d1ffcef765ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:49.237952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822092c-5d2b-45fc-87dd-d1ffcef765ec admin admin] PATH_INFO: `/v3/projects/98cd7da92abf4da3a76b4b278c7dc597/users/976e0488f3e7473e848ab529ea8362b2/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:49.241391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822092c-5d2b-45fc-87dd-d1ffcef765ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cd7da92abf4da3a76b4b278c7dc597, user_id=976e0488f3e7473e848ab529ea8362b2, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:49.244343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822092c-5d2b-45fc-87dd-d1ffcef765ec admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:49.257809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 902/1804] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:49 2022] PUT /identity/v3/projects/98cd7da92abf4da3a76b4b278c7dc597/users/976e0488f3e7473e848ab529ea8362b2/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:49.270340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02557f0-9e8b-4f24-8e5d-0b2767117f3f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:49.270657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02557f0-9e8b-4f24-8e5d-0b2767117f3f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:49.271014 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02557f0-9e8b-4f24-8e5d-0b2767117f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:49.306929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e02557f0-9e8b-4f24-8e5d-0b2767117f3f None None] MFA Rules not processed for user `976e0488f3e7473e848ab529ea8362b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:49.308702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e02557f0-9e8b-4f24-8e5d-0b2767117f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:49.390801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:49.400955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822092c-5d2b-45fc-87dd-d1ffcef765ec admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:49.402804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d822092c-5d2b-45fc-87dd-d1ffcef765ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:49.430506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 903/1805] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:49.447776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02557f0-9e8b-4f24-8e5d-0b2767117f3f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:49.449601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e02557f0-9e8b-4f24-8e5d-0b2767117f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:49.467613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:49.503152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822092c-5d2b-45fc-87dd-d1ffcef765ec admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:49.519824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e9575-bf7d-4b16-a64a-b700addaeb16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:49.520325 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e9575-bf7d-4b16-a64a-b700addaeb16 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:49.521066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e9575-bf7d-4b16-a64a-b700addaeb16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:49.521504 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e9575-bf7d-4b16-a64a-b700addaeb16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:49.524330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-bd1e9575-bf7d-4b16-a64a-b700addaeb16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:49.542997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02557f0-9e8b-4f24-8e5d-0b2767117f3f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:49.547016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc92518-0dc8-4213-90da-7370b0c2f5c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:49.547784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc92518-0dc8-4213-90da-7370b0c2f5c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:49.548139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc92518-0dc8-4213-90da-7370b0c2f5c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:49.548671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc92518-0dc8-4213-90da-7370b0c2f5c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:49.562690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc92518-0dc8-4213-90da-7370b0c2f5c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:49.564889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc92518-0dc8-4213-90da-7370b0c2f5c6 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:49.579276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 904/1806] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:49 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:05:49.637577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e9575-bf7d-4b16-a64a-b700addaeb16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:49.639850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e9575-bf7d-4b16-a64a-b700addaeb16 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:49.665589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 903/1807] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:49.815343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc92518-0dc8-4213-90da-7370b0c2f5c6 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:49.827890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc92518-0dc8-4213-90da-7370b0c2f5c6 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:49.832305 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1225288-c3ec-4e22-9764-e11298093dea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:49.834069 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1225288-c3ec-4e22-9764-e11298093dea service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:49.834558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1225288-c3ec-4e22-9764-e11298093dea service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:49.837093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1225288-c3ec-4e22-9764-e11298093dea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:49.908386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4f926-0ae8-4378-a550-20d6058319f1 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:49.908828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4f926-0ae8-4378-a550-20d6058319f1 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:49.909257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4f926-0ae8-4378-a550-20d6058319f1 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:49.911471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 904/1808] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 08 02:05:49.929504 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68639168-f041-40f7-9f51-3dd97e2ee317 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:49.929921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68639168-f041-40f7-9f51-3dd97e2ee317 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:49.930102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68639168-f041-40f7-9f51-3dd97e2ee317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:49.963095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-68639168-f041-40f7-9f51-3dd97e2ee317 None None] MFA Rules not processed for user `4eb69af145154c6d85bd4b845979e6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:49.966679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-68639168-f041-40f7-9f51-3dd97e2ee317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:50.021816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1225288-c3ec-4e22-9764-e11298093dea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:50.023473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1225288-c3ec-4e22-9764-e11298093dea service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:50.075422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 905/1809] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:50.077856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1225288-c3ec-4e22-9764-e11298093dea service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:50.085862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c1225288-c3ec-4e22-9764-e11298093dea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:50.169744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1225288-c3ec-4e22-9764-e11298093dea service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:50.175662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:50.177767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:50.178118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:50.178530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] PATH_INFO: `/v3/users/ec488621fe5a4d31aa2be1204c239d82` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:50.179960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec488621fe5a4d31aa2be1204c239d82)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:50.182912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:50.217209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 905/1810] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:05:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:50.225669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68639168-f041-40f7-9f51-3dd97e2ee317 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:50.226532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:50.242070 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68639168-f041-40f7-9f51-3dd97e2ee317 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:50.253320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48831a8d-dd5d-48f3-a800-9326e8030640 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:50.254051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48831a8d-dd5d-48f3-a800-9326e8030640 service placement] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:50.254520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48831a8d-dd5d-48f3-a800-9326e8030640 service placement] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:50.254968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48831a8d-dd5d-48f3-a800-9326e8030640 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:50.267064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec488621fe5a4d31aa2be1204c239d82'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:50.268109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec488621fe5a4d31aa2be1204c239d82'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:50.269000 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48831a8d-dd5d-48f3-a800-9326e8030640 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:50.271462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48831a8d-dd5d-48f3-a800-9326e8030640 service placement] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:50.280201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec488621fe5a4d31aa2be1204c239d82'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:50.281214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec488621fe5a4d31aa2be1204c239d82'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:50.288828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 906/1811] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:05:50.292879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec488621fe5a4d31aa2be1204c239d82'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:50.336093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 906/1812] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:50 2022] DELETE /identity/v3/users/ec488621fe5a4d31aa2be1204c239d82 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:50.751785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48831a8d-dd5d-48f3-a800-9326e8030640 service placement] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:50.771223 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48831a8d-dd5d-48f3-a800-9326e8030640 service placement] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:50.884173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:50.885304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:50.885588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:50.886055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] PATH_INFO: `/v3/projects/c75a902d9e5b4fdf9b423d579f900e02` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:50.887256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75a902d9e5b4fdf9b423d579f900e02)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:50.889428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:50.920515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:50.991636 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75a902d9e5b4fdf9b423d579f900e02', 'request_id': 'req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:51.000139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75a902d9e5b4fdf9b423d579f900e02', 'request_id': 'req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:51.007921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75a902d9e5b4fdf9b423d579f900e02', 'request_id': 'req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:51.027732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] The token cache is being invalidate because project c75a902d9e5b4fdf9b423d579f900e02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:51.028226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:51.028936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:51.030730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 907/1813] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:50 2022] DELETE /identity/v3/projects/c75a902d9e5b4fdf9b423d579f900e02 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:51.034712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:51.046146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:51.061748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c26378-47a3-4471-b0f3-64ffb4cedde8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:51.126944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:51.127882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:51.128169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:51.128540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] PATH_INFO: `/v3/users/e406d8eae1c6468a89a90d86fb672441` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:51.129893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e406d8eae1c6468a89a90d86fb672441)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:51.132259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:51.200994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e406d8eae1c6468a89a90d86fb672441'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:51.201568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e406d8eae1c6468a89a90d86fb672441'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:51.215005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e406d8eae1c6468a89a90d86fb672441'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:51.215718 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e406d8eae1c6468a89a90d86fb672441'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:51.225010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7ef96dfa-9dd0-43df-95a1-ef05583b9736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e406d8eae1c6468a89a90d86fb672441'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:51.244951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 907/1814] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:51 2022] DELETE /identity/v3/users/e406d8eae1c6468a89a90d86fb672441 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:05:52.390242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:52.397403 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:52.467194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1aa6eb-6eee-4eb0-8529-2ac26dfe68ce admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:52.644860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:52.646018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:52.646387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:52.646680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] PATH_INFO: `/v3/projects/9043fd9359424b3aba901331410be340` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:52.650485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9043fd9359424b3aba901331410be340)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:52.655288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:52.799276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9043fd9359424b3aba901331410be340', 'request_id': 'req-cdf3455f-d56d-4859-84ec-79d8be0c26c3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:52.821859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9043fd9359424b3aba901331410be340', 'request_id': 'req-cdf3455f-d56d-4859-84ec-79d8be0c26c3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:52.839442 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9043fd9359424b3aba901331410be340', 'request_id': 'req-cdf3455f-d56d-4859-84ec-79d8be0c26c3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:52.870902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] The token cache is being invalidate because project 9043fd9359424b3aba901331410be340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:52.871212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:52.872860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-cdf3455f-d56d-4859-84ec-79d8be0c26c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:52.884798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 908/1815] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:52 2022] DELETE /identity/v3/projects/9043fd9359424b3aba901331410be340 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:52.886729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900919-7636-405a-a728-6d161dba998e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:52.887210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900919-7636-405a-a728-6d161dba998e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:52.887461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900919-7636-405a-a728-6d161dba998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:52.944065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-22900919-7636-405a-a728-6d161dba998e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:52.946458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-22900919-7636-405a-a728-6d161dba998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:52.965607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:53.053714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 908/1816] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:53.054393 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140a0d6-39b6-4558-ad8b-c65622fa3b90 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.054671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140a0d6-39b6-4558-ad8b-c65622fa3b90 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.054957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140a0d6-39b6-4558-ad8b-c65622fa3b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.083543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3140a0d6-39b6-4558-ad8b-c65622fa3b90 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:53.085044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3140a0d6-39b6-4558-ad8b-c65622fa3b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:53.102316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 909/1817] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:53.110934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900919-7636-405a-a728-6d161dba998e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:53.137094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900919-7636-405a-a728-6d161dba998e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:53.149436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df415a5c-0d7d-44bb-b5c3-76996f008e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:53.150792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df415a5c-0d7d-44bb-b5c3-76996f008e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.151363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df415a5c-0d7d-44bb-b5c3-76996f008e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.153702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df415a5c-0d7d-44bb-b5c3-76996f008e09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.154753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df415a5c-0d7d-44bb-b5c3-76996f008e09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:53.155274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df415a5c-0d7d-44bb-b5c3-76996f008e09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:53.157272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df415a5c-0d7d-44bb-b5c3-76996f008e09 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:53.179752 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 909/1818] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:53.181748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140a0d6-39b6-4558-ad8b-c65622fa3b90 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:53.195482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140a0d6-39b6-4558-ad8b-c65622fa3b90 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:53.201807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfe673-7e68-4c7f-94d1-8b598a818f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:53.202781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe673-7e68-4c7f-94d1-8b598a818f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.203273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe673-7e68-4c7f-94d1-8b598a818f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.203752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe673-7e68-4c7f-94d1-8b598a818f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.204773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfe673-7e68-4c7f-94d1-8b598a818f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:53.207547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfe673-7e68-4c7f-94d1-8b598a818f23 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:53.246456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df415a5c-0d7d-44bb-b5c3-76996f008e09 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:53.252581 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 910/1819] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:53 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:53.267835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df415a5c-0d7d-44bb-b5c3-76996f008e09 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:53.402717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f810af-d3d6-49eb-9fb7-94e946bcded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:53.403695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f810af-d3d6-49eb-9fb7-94e946bcded4 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.404121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f810af-d3d6-49eb-9fb7-94e946bcded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.404575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f810af-d3d6-49eb-9fb7-94e946bcded4 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.405429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f810af-d3d6-49eb-9fb7-94e946bcded4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:53.408787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f810af-d3d6-49eb-9fb7-94e946bcded4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:53.448515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dabe88-37e7-4775-bee9-d42f848ec66c None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.452997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dabe88-37e7-4775-bee9-d42f848ec66c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.454300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dabe88-37e7-4775-bee9-d42f848ec66c None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.459689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 910/1820] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:53 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:05:53.464359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 911/1821] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:05:53 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:53.465125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f810af-d3d6-49eb-9fb7-94e946bcded4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:53.476547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77bfb4-15d6-43b8-8a3f-a9f55106253b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.477191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77bfb4-15d6-43b8-8a3f-a9f55106253b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.477756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77bfb4-15d6-43b8-8a3f-a9f55106253b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.497574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f810af-d3d6-49eb-9fb7-94e946bcded4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:53.498598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:53.518223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb2156-9c77-47bf-83bf-a70dd373a779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:53.518854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb2156-9c77-47bf-83bf-a70dd373a779 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.519143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb2156-9c77-47bf-83bf-a70dd373a779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.519422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb2156-9c77-47bf-83bf-a70dd373a779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.520424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb2156-9c77-47bf-83bf-a70dd373a779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:53.522511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb2156-9c77-47bf-83bf-a70dd373a779 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:53.530300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-5c77bfb4-15d6-43b8-8a3f-a9f55106253b None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:53.540333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5c77bfb4-15d6-43b8-8a3f-a9f55106253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:53.558615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 911/1822] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:05:53.559823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb2156-9c77-47bf-83bf-a70dd373a779 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:53.586312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb2156-9c77-47bf-83bf-a70dd373a779 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:53.599126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de96f0-550c-4411-b46d-5468eab62db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:53.599945 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de96f0-550c-4411-b46d-5468eab62db2 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.600237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de96f0-550c-4411-b46d-5468eab62db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.600548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de96f0-550c-4411-b46d-5468eab62db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.601290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de96f0-550c-4411-b46d-5468eab62db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:53.605064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de96f0-550c-4411-b46d-5468eab62db2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:53.631954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 912/1823] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:53.636355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de96f0-550c-4411-b46d-5468eab62db2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:53.654926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de96f0-550c-4411-b46d-5468eab62db2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:53.671933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da9bac-5be0-4b05-8fbe-76cebe490356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:53.672970 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da9bac-5be0-4b05-8fbe-76cebe490356 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.673429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da9bac-5be0-4b05-8fbe-76cebe490356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.684123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da9bac-5be0-4b05-8fbe-76cebe490356 admin admin] PATH_INFO: `/v3/projects/6a03b2c5b4404f2d918b010c69eee729/users/925eb18fd15e4853bc4b15597419ae8b/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.729476 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da9bac-5be0-4b05-8fbe-76cebe490356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a03b2c5b4404f2d918b010c69eee729, user_id=925eb18fd15e4853bc4b15597419ae8b, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:53.732855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da9bac-5be0-4b05-8fbe-76cebe490356 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:53.750989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 912/1824] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:05:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:53.758340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 913/1825] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:53 2022] PUT /identity/v3/projects/6a03b2c5b4404f2d918b010c69eee729/users/925eb18fd15e4853bc4b15597419ae8b/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:53.769156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618588e1-37ce-4dc0-be00-4107a2769d67 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.769156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618588e1-37ce-4dc0-be00-4107a2769d67 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.769361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618588e1-37ce-4dc0-be00-4107a2769d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.779154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da9bac-5be0-4b05-8fbe-76cebe490356 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:53.781052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-06da9bac-5be0-4b05-8fbe-76cebe490356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:53.842219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-618588e1-37ce-4dc0-be00-4107a2769d67 None None] MFA Rules not processed for user `925eb18fd15e4853bc4b15597419ae8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:53.844933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-618588e1-37ce-4dc0-be00-4107a2769d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:53.898334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:53.948655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 913/1826] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:53.949828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618588e1-37ce-4dc0-be00-4107a2769d67 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:53.952963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da9bac-5be0-4b05-8fbe-76cebe490356 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:53.961593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848edcd-24c2-4a8b-856a-3fcc61c65fca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:53.963180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848edcd-24c2-4a8b-856a-3fcc61c65fca service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:53.963476 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848edcd-24c2-4a8b-856a-3fcc61c65fca service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:53.963760 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848edcd-24c2-4a8b-856a-3fcc61c65fca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:53.977340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848edcd-24c2-4a8b-856a-3fcc61c65fca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:53.979057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848edcd-24c2-4a8b-856a-3fcc61c65fca service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:53.994208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 914/1827] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:05:54.032625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618588e1-37ce-4dc0-be00-4107a2769d67 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:54.040234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f38a-061e-4ed0-92a6-f8f63a08069c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:54.041056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f38a-061e-4ed0-92a6-f8f63a08069c admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.041585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f38a-061e-4ed0-92a6-f8f63a08069c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.041902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f38a-061e-4ed0-92a6-f8f63a08069c admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.042621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36f38a-061e-4ed0-92a6-f8f63a08069c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:54.045242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36f38a-061e-4ed0-92a6-f8f63a08069c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:54.078752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 914/1828] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:05:53 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:54.082514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848edcd-24c2-4a8b-856a-3fcc61c65fca service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:54.094910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848edcd-24c2-4a8b-856a-3fcc61c65fca service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:54.154335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f38a-061e-4ed0-92a6-f8f63a08069c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:54.156036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6b36f38a-061e-4ed0-92a6-f8f63a08069c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:54.165775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f36e1-7545-4eca-9a8f-622e7eb84189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:54.166739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f36e1-7545-4eca-9a8f-622e7eb84189 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.166915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f36e1-7545-4eca-9a8f-622e7eb84189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.167191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f36e1-7545-4eca-9a8f-622e7eb84189 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.167880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f36e1-7545-4eca-9a8f-622e7eb84189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:54.169851 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f36e1-7545-4eca-9a8f-622e7eb84189 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:54.170422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:54.197797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 915/1829] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:05:54 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:05:54.201813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f36e1-7545-4eca-9a8f-622e7eb84189 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:54.212150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f36e1-7545-4eca-9a8f-622e7eb84189 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:54.217271 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c8a98-6b95-450c-8778-67205fc85a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:54.218087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c8a98-6b95-450c-8778-67205fc85a72 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.218306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c8a98-6b95-450c-8778-67205fc85a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.218592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c8a98-6b95-450c-8778-67205fc85a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.219330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c8a98-6b95-450c-8778-67205fc85a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:54.221153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c8a98-6b95-450c-8778-67205fc85a72 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:54.232963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 916/1830] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:05:54.237042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c8a98-6b95-450c-8778-67205fc85a72 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:54.240008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f38a-061e-4ed0-92a6-f8f63a08069c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:54.244735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:54.245615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.245901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.246180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] PATH_INFO: `/v3/users/7725e1370a3e4e3f8e19fc6a00ee7a64` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.247385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7725e1370a3e4e3f8e19fc6a00ee7a64)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:54.247900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c8a98-6b95-450c-8778-67205fc85a72 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:54.249431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:54.252696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d96a9-9931-4284-ad75-3a4e631120e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:54.253580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d96a9-9931-4284-ad75-3a4e631120e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.253853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d96a9-9931-4284-ad75-3a4e631120e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.254132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d96a9-9931-4284-ad75-3a4e631120e8 admin admin] PATH_INFO: `/v3/projects/07b75cf7b7ec4f4abd3df4d8184367b9/users/b445abde73514fa497fe9cdbc16d5c5e/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.269872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d96a9-9931-4284-ad75-3a4e631120e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b75cf7b7ec4f4abd3df4d8184367b9, user_id=b445abde73514fa497fe9cdbc16d5c5e, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:54.272517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d96a9-9931-4284-ad75-3a4e631120e8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:54.288278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 917/1831] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:54 2022] PUT /identity/v3/projects/07b75cf7b7ec4f4abd3df4d8184367b9/users/b445abde73514fa497fe9cdbc16d5c5e/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:05:54.292806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d96a9-9931-4284-ad75-3a4e631120e8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:54.300097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7725e1370a3e4e3f8e19fc6a00ee7a64'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:54.300857 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7725e1370a3e4e3f8e19fc6a00ee7a64'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:54.308957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7725e1370a3e4e3f8e19fc6a00ee7a64'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:54.309633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7725e1370a3e4e3f8e19fc6a00ee7a64'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:54.322997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2b53514d-c276-4746-a9f0-e2b27355d418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7725e1370a3e4e3f8e19fc6a00ee7a64'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:54.338341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 915/1832] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:54 2022] DELETE /identity/v3/users/7725e1370a3e4e3f8e19fc6a00ee7a64 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:05:54.342904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924964ba-358d-4f52-9e4d-be70423c3f71 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.343194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924964ba-358d-4f52-9e4d-be70423c3f71 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.343473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924964ba-358d-4f52-9e4d-be70423c3f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.360513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d96a9-9931-4284-ad75-3a4e631120e8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:54.362113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-924964ba-358d-4f52-9e4d-be70423c3f71 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:54.363347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-924964ba-358d-4f52-9e4d-be70423c3f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:54.402120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:54.438714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44704ef2-e1f7-4332-aa4a-064f4e4abbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:54.439359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44704ef2-e1f7-4332-aa4a-064f4e4abbbe admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.439644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44704ef2-e1f7-4332-aa4a-064f4e4abbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.439925 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44704ef2-e1f7-4332-aa4a-064f4e4abbbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.440710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44704ef2-e1f7-4332-aa4a-064f4e4abbbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:54.441736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 916/1833] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:54.442470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44704ef2-e1f7-4332-aa4a-064f4e4abbbe admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:54.453483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924964ba-358d-4f52-9e4d-be70423c3f71 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:54.454893 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-924964ba-358d-4f52-9e4d-be70423c3f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:54.455772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 918/1834] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:05:54.460211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44704ef2-e1f7-4332-aa4a-064f4e4abbbe admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:54.461906 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-44704ef2-e1f7-4332-aa4a-064f4e4abbbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:54.475815 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44704ef2-e1f7-4332-aa4a-064f4e4abbbe admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:54.480279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d2b27-766e-44ac-9959-3fac45f74ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:54.481233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d2b27-766e-44ac-9959-3fac45f74ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.481518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d2b27-766e-44ac-9959-3fac45f74ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.481792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d2b27-766e-44ac-9959-3fac45f74ac0 admin admin] PATH_INFO: `/v3/projects/07b75cf7b7ec4f4abd3df4d8184367b9/users/b445abde73514fa497fe9cdbc16d5c5e/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.483772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d2b27-766e-44ac-9959-3fac45f74ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b75cf7b7ec4f4abd3df4d8184367b9, user_id=b445abde73514fa497fe9cdbc16d5c5e, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:54.486460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d2b27-766e-44ac-9959-3fac45f74ac0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:54.493132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:54.499033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 919/1835] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:05:54 2022] PUT /identity/v3/projects/07b75cf7b7ec4f4abd3df4d8184367b9/users/b445abde73514fa497fe9cdbc16d5c5e/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:05:54.513986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2ec71-f917-4ad4-ad6d-680d1b97191f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.514553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2ec71-f917-4ad4-ad6d-680d1b97191f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.514738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2ec71-f917-4ad4-ad6d-680d1b97191f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.534441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924964ba-358d-4f52-9e4d-be70423c3f71 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:54.561910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ddb2ec71-f917-4ad4-ad6d-680d1b97191f None None] MFA Rules not processed for user `b445abde73514fa497fe9cdbc16d5c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:54.563136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ddb2ec71-f917-4ad4-ad6d-680d1b97191f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:54.604123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6fd6a-667f-4dd7-9ab1-7feb49a87ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:54.604842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6fd6a-667f-4dd7-9ab1-7feb49a87ade service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.605125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6fd6a-667f-4dd7-9ab1-7feb49a87ade service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.605399 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6fd6a-667f-4dd7-9ab1-7feb49a87ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.652445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 920/1836] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:54.666187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2ec71-f917-4ad4-ad6d-680d1b97191f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:54.667665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ddb2ec71-f917-4ad4-ad6d-680d1b97191f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:54.675860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6fd6a-667f-4dd7-9ab1-7feb49a87ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:54.677936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6fd6a-667f-4dd7-9ab1-7feb49a87ade service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:54.687492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 917/1837] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:54.729100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2ec71-f917-4ad4-ad6d-680d1b97191f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:54.732963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307a0b2-971c-420c-8459-d5cd080b7dd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:54.733616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307a0b2-971c-420c-8459-d5cd080b7dd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:54.733899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307a0b2-971c-420c-8459-d5cd080b7dd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:54.734360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307a0b2-971c-420c-8459-d5cd080b7dd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:54.745509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307a0b2-971c-420c-8459-d5cd080b7dd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:54.747208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307a0b2-971c-420c-8459-d5cd080b7dd7 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:54.759794 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 921/1838] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:05:54.994410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6fd6a-667f-4dd7-9ab1-7feb49a87ade service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:54.995883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-72f6fd6a-667f-4dd7-9ab1-7feb49a87ade service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:55.008943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6fd6a-667f-4dd7-9ab1-7feb49a87ade service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:55.017462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:55.018352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:55.018641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:55.018920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] PATH_INFO: `/v3/projects/5092352dc53c4014a067160b260b2132` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:55.020190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5092352dc53c4014a067160b260b2132)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:55.022485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:55.075842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5092352dc53c4014a067160b260b2132', 'request_id': 'req-56e71650-4f9f-494e-b9fd-9d6b72ac246b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.081544 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5092352dc53c4014a067160b260b2132', 'request_id': 'req-56e71650-4f9f-494e-b9fd-9d6b72ac246b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.088465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5092352dc53c4014a067160b260b2132', 'request_id': 'req-56e71650-4f9f-494e-b9fd-9d6b72ac246b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.103367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] The token cache is being invalidate because project 5092352dc53c4014a067160b260b2132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:55.103661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.104441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.106008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 918/1839] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:54 2022] DELETE /identity/v3/projects/5092352dc53c4014a067160b260b2132 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:55.109991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307a0b2-971c-420c-8459-d5cd080b7dd7 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:55.123443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307a0b2-971c-420c-8459-d5cd080b7dd7 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:55.171828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:55.185546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:55.186243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:55.186664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:55.187040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] PATH_INFO: `/v3/users/f00a8fe38a204a829b434fa09f544783` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:55.188528 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00a8fe38a204a829b434fa09f544783)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:55.190620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:55.217853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:55.219165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:55.234570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e71650-4f9f-494e-b9fd-9d6b72ac246b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:55.239619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:55.240937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:55.241229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:55.241579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] PATH_INFO: `/v3/users/925eb18fd15e4853bc4b15597419ae8b` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:55.243044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] RBAC: Authorizing `identity:delete_user(user_id=925eb18fd15e4853bc4b15597419ae8b)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:55.245172 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:55.260135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00a8fe38a204a829b434fa09f544783'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.262033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:55.262608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00a8fe38a204a829b434fa09f544783'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.272900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00a8fe38a204a829b434fa09f544783'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.273617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00a8fe38a204a829b434fa09f544783'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.282715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d4204487-ebc6-4d4d-8125-207d903c7a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00a8fe38a204a829b434fa09f544783'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.308258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 922/1840] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:55 2022] DELETE /identity/v3/users/f00a8fe38a204a829b434fa09f544783 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:55.315012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '925eb18fd15e4853bc4b15597419ae8b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.315681 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '925eb18fd15e4853bc4b15597419ae8b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.323162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '925eb18fd15e4853bc4b15597419ae8b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.323861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '925eb18fd15e4853bc4b15597419ae8b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.338080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '925eb18fd15e4853bc4b15597419ae8b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:55.352913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 919/1841] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:55 2022] DELETE /identity/v3/users/925eb18fd15e4853bc4b15597419ae8b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:55.357867 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd6222-ba07-4ef5-8780-a68ebf24824c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:55.358194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd6222-ba07-4ef5-8780-a68ebf24824c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:55.358506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd6222-ba07-4ef5-8780-a68ebf24824c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:55.376559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d2cd6222-ba07-4ef5-8780-a68ebf24824c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:55.377787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d2cd6222-ba07-4ef5-8780-a68ebf24824c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:55.441682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 923/1842] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:55.451971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:55.453546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:55.519379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba2057-071b-4731-8bb9-65fb5667828f admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:55.523767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0e09e-6410-4356-a179-18604fd51096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:55.524450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0e09e-6410-4356-a179-18604fd51096 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:55.524830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0e09e-6410-4356-a179-18604fd51096 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:55.525184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0e09e-6410-4356-a179-18604fd51096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:55.536021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0e09e-6410-4356-a179-18604fd51096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:55.538477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0e09e-6410-4356-a179-18604fd51096 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:55.549577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 920/1843] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:56.793451 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd6222-ba07-4ef5-8780-a68ebf24824c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:56.796803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d2cd6222-ba07-4ef5-8780-a68ebf24824c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:56.845558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd6222-ba07-4ef5-8780-a68ebf24824c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:56.867787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:56.869156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:56.869563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:56.869851 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] PATH_INFO: `/v3/projects/fde2705e191a41b1914b97cc8ef0b0c1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:56.873845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde2705e191a41b1914b97cc8ef0b0c1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:56.875620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0e09e-6410-4356-a179-18604fd51096 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:56.878601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-12a0e09e-6410-4356-a179-18604fd51096 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:56.890202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:56.909786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0e09e-6410-4356-a179-18604fd51096 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:56.938817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:56.940820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:56.941593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:56.941872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] PATH_INFO: `/v3/users/6aa7db022b12421a956f9f0f5e52cbd9` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:56.944818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa7db022b12421a956f9f0f5e52cbd9)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:56.956274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:57.071203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde2705e191a41b1914b97cc8ef0b0c1', 'request_id': 'req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.083758 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde2705e191a41b1914b97cc8ef0b0c1', 'request_id': 'req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.108504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde2705e191a41b1914b97cc8ef0b0c1', 'request_id': 'req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.112150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:57.128750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] The token cache is being invalidate because project fde2705e191a41b1914b97cc8ef0b0c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:57.129153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.131843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.135052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 924/1844] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:56 2022] DELETE /identity/v3/projects/fde2705e191a41b1914b97cc8ef0b0c1 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:57.137787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:57.139370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:57.167975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa7db022b12421a956f9f0f5e52cbd9'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.169306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa7db022b12421a956f9f0f5e52cbd9'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.200966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa7db022b12421a956f9f0f5e52cbd9'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.202719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa7db022b12421a956f9f0f5e52cbd9'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.256616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa7db022b12421a956f9f0f5e52cbd9'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.288808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 921/1845] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:56 2022] DELETE /identity/v3/users/6aa7db022b12421a956f9f0f5e52cbd9 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:57.289672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:57.293813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:57.322000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1d7d5-f5f4-4a23-9ce0-cc07230ed779 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:57.323337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:57.491516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83ff82-32b0-4307-8849-6c2736f64df7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:57.502895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:57.504454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:57.504988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:57.505314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:57.505993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] PATH_INFO: `/v3/users/2e7c8da56446429881ef241d2ce72538` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:57.506326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:57.506617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:57.507784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] PATH_INFO: `/v3/users/696147cd85ed4e0586d6b9673ef991a0` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:57.508846 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7c8da56446429881ef241d2ce72538)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:57.510412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=696147cd85ed4e0586d6b9673ef991a0)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:57.512883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:57.514569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:57.591470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7c8da56446429881ef241d2ce72538'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.592595 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7c8da56446429881ef241d2ce72538'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.604473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7c8da56446429881ef241d2ce72538'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.606000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7c8da56446429881ef241d2ce72538'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.618280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7c8da56446429881ef241d2ce72538'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.623165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '696147cd85ed4e0586d6b9673ef991a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.623917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '696147cd85ed4e0586d6b9673ef991a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.637188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '696147cd85ed4e0586d6b9673ef991a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.638177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '696147cd85ed4e0586d6b9673ef991a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.647771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 925/1846] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:57 2022] DELETE /identity/v3/users/2e7c8da56446429881ef241d2ce72538 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:57.658980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-43bf45c8-b787-4fca-b875-367794ebe707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '696147cd85ed4e0586d6b9673ef991a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:57.680452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:57.685635 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:57.687502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 922/1847] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:57 2022] DELETE /identity/v3/users/696147cd85ed4e0586d6b9673ef991a0 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:57.689066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248a2e8-1504-4d1c-9ffa-73c26e66efc3 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:57.689372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248a2e8-1504-4d1c-9ffa-73c26e66efc3 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:57.689675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248a2e8-1504-4d1c-9ffa-73c26e66efc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:57.713010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-0248a2e8-1504-4d1c-9ffa-73c26e66efc3 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:57.714367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0248a2e8-1504-4d1c-9ffa-73c26e66efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:57.744720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:57.842755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 923/1848] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:57.843769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d59ac-4c8f-47ff-9444-cf2ee77bc9d9 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:57.844064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d59ac-4c8f-47ff-9444-cf2ee77bc9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:57.844349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d59ac-4c8f-47ff-9444-cf2ee77bc9d9 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:57.854706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67b21e-723c-4792-92b1-1e8329292b10 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:57.855473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 924/1849] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:05:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:05:57.857055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a7aef-947e-4a38-aa82-1f2ce7ff5c2c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:57.857351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a7aef-947e-4a38-aa82-1f2ce7ff5c2c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:57.857779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a7aef-947e-4a38-aa82-1f2ce7ff5c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:57.870369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:57.871199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:57.871492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:57.871772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] PATH_INFO: `/v3/projects/6a03b2c5b4404f2d918b010c69eee729` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:57.876582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a03b2c5b4404f2d918b010c69eee729)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:57.878790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:57.897179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-119a7aef-947e-4a38-aa82-1f2ce7ff5c2c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:57.898574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-119a7aef-947e-4a38-aa82-1f2ce7ff5c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:57.909898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 925/1850] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:57.912104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff0895-af39-4595-9717-c26c93d84b61 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:57.912410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff0895-af39-4595-9717-c26c93d84b61 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:57.912688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff0895-af39-4595-9717-c26c93d84b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:57.938228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-47ff0895-af39-4595-9717-c26c93d84b61 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:57.939723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-47ff0895-af39-4595-9717-c26c93d84b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:57.963370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:57.996521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a03b2c5b4404f2d918b010c69eee729', 'request_id': 'req-e079eb55-c5d8-4052-b643-ad5175fa1cf7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:58.013060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a03b2c5b4404f2d918b010c69eee729', 'request_id': 'req-e079eb55-c5d8-4052-b643-ad5175fa1cf7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:58.031697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a03b2c5b4404f2d918b010c69eee729', 'request_id': 'req-e079eb55-c5d8-4052-b643-ad5175fa1cf7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:58.055745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] The token cache is being invalidate because project 6a03b2c5b4404f2d918b010c69eee729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:58.056312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:58.058394 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:58.061265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 926/1851] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:57 2022] DELETE /identity/v3/projects/6a03b2c5b4404f2d918b010c69eee729 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:58.063895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:58.069756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:58.117506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 926/1852] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:05:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:58.118711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff0895-af39-4595-9717-c26c93d84b61 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:58.122959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-47ff0895-af39-4595-9717-c26c93d84b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:58.175001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079eb55-c5d8-4052-b643-ad5175fa1cf7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:58.180512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82067546-6b34-4898-90c6-2fb99310d7b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:58.181915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82067546-6b34-4898-90c6-2fb99310d7b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:58.182358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82067546-6b34-4898-90c6-2fb99310d7b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:58.182814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82067546-6b34-4898-90c6-2fb99310d7b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:58.185431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-82067546-6b34-4898-90c6-2fb99310d7b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:58.238608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff0895-af39-4595-9717-c26c93d84b61 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:58.250617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71be11-9df2-4345-b286-43d223c16665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:58.252643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71be11-9df2-4345-b286-43d223c16665 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:58.253143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71be11-9df2-4345-b286-43d223c16665 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:58.254229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71be11-9df2-4345-b286-43d223c16665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:58.267101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-eb71be11-9df2-4345-b286-43d223c16665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:58.285016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82067546-6b34-4898-90c6-2fb99310d7b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:58.287223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82067546-6b34-4898-90c6-2fb99310d7b4 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:58.291892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:58.300612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 927/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:58.301319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71be11-9df2-4345-b286-43d223c16665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:58.302724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82067546-6b34-4898-90c6-2fb99310d7b4 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:58.304566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71be11-9df2-4345-b286-43d223c16665 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:58.309452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-82067546-6b34-4898-90c6-2fb99310d7b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:58.317066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 927/1854] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:58.323567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82067546-6b34-4898-90c6-2fb99310d7b4 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:58.328991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:58.330072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:58.330359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:58.330642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] PATH_INFO: `/v3/users/b445abde73514fa497fe9cdbc16d5c5e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:58.334538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b445abde73514fa497fe9cdbc16d5c5e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:58.337611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:58.397769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71be11-9df2-4345-b286-43d223c16665 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:58.402002 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-eb71be11-9df2-4345-b286-43d223c16665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:58.414762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b445abde73514fa497fe9cdbc16d5c5e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:58.415822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b445abde73514fa497fe9cdbc16d5c5e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:58.425830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b445abde73514fa497fe9cdbc16d5c5e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:58.427318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b445abde73514fa497fe9cdbc16d5c5e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:58.438568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b445abde73514fa497fe9cdbc16d5c5e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:58.468422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 928/1855] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:58 2022] DELETE /identity/v3/users/b445abde73514fa497fe9cdbc16d5c5e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:58.495876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71be11-9df2-4345-b286-43d223c16665 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:58.559123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:58.586752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c62ba-2e53-42c9-9816-f5ea037df148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:58.587699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c62ba-2e53-42c9-9816-f5ea037df148 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:58.587952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c62ba-2e53-42c9-9816-f5ea037df148 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:58.588335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c62ba-2e53-42c9-9816-f5ea037df148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:58.592160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-128c62ba-2e53-42c9-9816-f5ea037df148 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:58.612477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c62ba-2e53-42c9-9816-f5ea037df148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:58.614648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c62ba-2e53-42c9-9816-f5ea037df148 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:58.628589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 928/1856] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:05:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:05:59.301698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:59.303425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:59.374125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c62ba-2e53-42c9-9816-f5ea037df148 service nova] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:59.376687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-128c62ba-2e53-42c9-9816-f5ea037df148 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:59.472167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54405cfc-c314-45da-8014-3e596dce95ef admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:59.479509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:59.482938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:59.484780 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:59.485143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] PATH_INFO: `/v3/projects/a88801a5139e40d0b4975a3213dfc8cf` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:59.489648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a88801a5139e40d0b4975a3213dfc8cf)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:59.491902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:59.516109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c62ba-2e53-42c9-9816-f5ea037df148 service nova] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:59.529501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:59.540836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:59.541130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:59.541419 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] PATH_INFO: `/v3/projects/77f347909002479584583958dc800b69` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:59.543020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f347909002479584583958dc800b69)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:59.545660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:59.594158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a88801a5139e40d0b4975a3213dfc8cf', 'request_id': 'req-fe7a2348-9487-47fd-a8a4-f479f04cebf0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.605225 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a88801a5139e40d0b4975a3213dfc8cf', 'request_id': 'req-fe7a2348-9487-47fd-a8a4-f479f04cebf0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.625417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a88801a5139e40d0b4975a3213dfc8cf', 'request_id': 'req-fe7a2348-9487-47fd-a8a4-f479f04cebf0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.641568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:59.648838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] The token cache is being invalidate because project a88801a5139e40d0b4975a3213dfc8cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:59.649481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.650874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f347909002479584583958dc800b69', 'request_id': 'req-f2e64157-e83f-4359-8690-9f57c1d83132'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.651551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.657997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 929/1857] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:59 2022] DELETE /identity/v3/projects/a88801a5139e40d0b4975a3213dfc8cf => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:59.658760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f347909002479584583958dc800b69', 'request_id': 'req-f2e64157-e83f-4359-8690-9f57c1d83132'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.658987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:59.661731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:59.668081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f347909002479584583958dc800b69', 'request_id': 'req-f2e64157-e83f-4359-8690-9f57c1d83132'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.699186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] The token cache is being invalidate because project 77f347909002479584583958dc800b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:05:59.699813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.700713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f2e64157-e83f-4359-8690-9f57c1d83132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.703915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 929/1858] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:59 2022] DELETE /identity/v3/projects/77f347909002479584583958dc800b69 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:05:59.714631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180ffa0-86d6-475a-a706-08f2a8b957bd None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:59.715079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180ffa0-86d6-475a-a706-08f2a8b957bd None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:59.715717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180ffa0-86d6-475a-a706-08f2a8b957bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:59.745321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-0180ffa0-86d6-475a-a706-08f2a8b957bd None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:59.746677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0180ffa0-86d6-475a-a706-08f2a8b957bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:59.782916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a2348-9487-47fd-a8a4-f479f04cebf0 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:59.855775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 930/1859] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:05:59.864334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b407-f051-4de7-aa3d-ba82926548f6 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:59.864793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b407-f051-4de7-aa3d-ba82926548f6 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:59.865445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b407-f051-4de7-aa3d-ba82926548f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:59.881800 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:59.882697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:59.882977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:59.883260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] PATH_INFO: `/v3/users/43f0cdd162bb4c968314755c5b886985` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:59.884641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] RBAC: Authorizing `identity:delete_user(user_id=43f0cdd162bb4c968314755c5b886985)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:59.887317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:59.896085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-9615b407-f051-4de7-aa3d-ba82926548f6 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:05:59.897776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-9615b407-f051-4de7-aa3d-ba82926548f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:05:59.916164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 931/1860] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:05:59.917495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615b407-f051-4de7-aa3d-ba82926548f6 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:59.922335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:05:59.933127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615b407-f051-4de7-aa3d-ba82926548f6 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:05:59.964880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a911c6-d267-46f3-b6bc-fd8bf2cc41ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:05:59.965783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a911c6-d267-46f3-b6bc-fd8bf2cc41ec admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:05:59.966899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a911c6-d267-46f3-b6bc-fd8bf2cc41ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:05:59.968389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a911c6-d267-46f3-b6bc-fd8bf2cc41ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:05:59.969215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a911c6-d267-46f3-b6bc-fd8bf2cc41ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:05:59.969542 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a911c6-d267-46f3-b6bc-fd8bf2cc41ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:05:59.972305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a911c6-d267-46f3-b6bc-fd8bf2cc41ec admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:05:59.988053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43f0cdd162bb4c968314755c5b886985'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.989186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43f0cdd162bb4c968314755c5b886985'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:05:59.995533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 932/1861] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:05:59.995533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a911c6-d267-46f3-b6bc-fd8bf2cc41ec admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:05:59.997153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f6a911c6-d267-46f3-b6bc-fd8bf2cc41ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.001780 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43f0cdd162bb4c968314755c5b886985'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.002523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43f0cdd162bb4c968314755c5b886985'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.016901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a911c6-d267-46f3-b6bc-fd8bf2cc41ec admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:00.028804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43f0cdd162bb4c968314755c5b886985'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.042949 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:00.043857 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.044435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.044885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] PATH_INFO: `/v3/projects/657dc367a1104ad6a5570a446ae39985` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.046252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=657dc367a1104ad6a5570a446ae39985)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:00.048351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:00.066085 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 930/1862] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:05:59 2022] DELETE /identity/v3/users/43f0cdd162bb4c968314755c5b886985 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:00.066085 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:00.070025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.122727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657dc367a1104ad6a5570a446ae39985', 'request_id': 'req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.127040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:00.131799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657dc367a1104ad6a5570a446ae39985', 'request_id': 'req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.140539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657dc367a1104ad6a5570a446ae39985', 'request_id': 'req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.168058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] The token cache is being invalidate because project 657dc367a1104ad6a5570a446ae39985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:00.168266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.169541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.176198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 933/1863] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:05:59 2022] DELETE /identity/v3/projects/657dc367a1104ad6a5570a446ae39985 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:00.176198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:00.176617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.193323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e61ae4-ff05-45c9-bdc8-a5799aa5565e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:00.298784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f920334-b9d4-4dba-934e-70cf1d076996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:00.300208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f920334-b9d4-4dba-934e-70cf1d076996 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.301083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f920334-b9d4-4dba-934e-70cf1d076996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.301796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f920334-b9d4-4dba-934e-70cf1d076996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.303454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f920334-b9d4-4dba-934e-70cf1d076996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:00.308166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f920334-b9d4-4dba-934e-70cf1d076996 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:00.339424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe609a4d-9ae9-4d7d-bd90-3e28832bea29 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:00.344890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:00.346280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.346867 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.347240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] PATH_INFO: `/v3/projects/07b75cf7b7ec4f4abd3df4d8184367b9` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.349473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b75cf7b7ec4f4abd3df4d8184367b9)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:00.352359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:00.366715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 931/1864] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:00 2022] POST /identity/v3/projects => generated 379 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:00.366715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f920334-b9d4-4dba-934e-70cf1d076996 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:00.368125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9f920334-b9d4-4dba-934e-70cf1d076996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.412435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b75cf7b7ec4f4abd3df4d8184367b9', 'request_id': 'req-9c484fe7-b9d5-4263-b48a-68ea78540ef8'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.418201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b75cf7b7ec4f4abd3df4d8184367b9', 'request_id': 'req-9c484fe7-b9d5-4263-b48a-68ea78540ef8'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.425164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b75cf7b7ec4f4abd3df4d8184367b9', 'request_id': 'req-9c484fe7-b9d5-4263-b48a-68ea78540ef8'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.446454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] The token cache is being invalidate because project 07b75cf7b7ec4f4abd3df4d8184367b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:00.447839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.448566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.450326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 934/1865] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:00 2022] DELETE /identity/v3/projects/07b75cf7b7ec4f4abd3df4d8184367b9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:00.450872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f920334-b9d4-4dba-934e-70cf1d076996 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:00.451310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:00.454801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.461123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:00.524849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:00.525739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.526018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.526290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] PATH_INFO: `/v3/users/5c647ddeb9c642d7aef7bd1fc99cef3c` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.527517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c647ddeb9c642d7aef7bd1fc99cef3c)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:00.529513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:00.533341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c484fe7-b9d5-4263-b48a-68ea78540ef8 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:00.534970 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:00.538755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50737e2-c8dd-4c1c-81e3-228e838f4127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:00.539505 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50737e2-c8dd-4c1c-81e3-228e838f4127 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.539879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50737e2-c8dd-4c1c-81e3-228e838f4127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.540224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50737e2-c8dd-4c1c-81e3-228e838f4127 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.541026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50737e2-c8dd-4c1c-81e3-228e838f4127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:00.543170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50737e2-c8dd-4c1c-81e3-228e838f4127 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:00.568642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 935/1866] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:00 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:00.570112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ad221-1390-42e1-a18f-b856b805a96b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.570506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ad221-1390-42e1-a18f-b856b805a96b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.570882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ad221-1390-42e1-a18f-b856b805a96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.590037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ab7ad221-1390-42e1-a18f-b856b805a96b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:00.591350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ab7ad221-1390-42e1-a18f-b856b805a96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.592233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c647ddeb9c642d7aef7bd1fc99cef3c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.592989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c647ddeb9c642d7aef7bd1fc99cef3c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.600319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 936/1867] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:00.601403 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ad221-1390-42e1-a18f-b856b805a96b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:00.602907 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ab7ad221-1390-42e1-a18f-b856b805a96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.605008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c647ddeb9c642d7aef7bd1fc99cef3c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.606864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c647ddeb9c642d7aef7bd1fc99cef3c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.616849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ad221-1390-42e1-a18f-b856b805a96b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:00.618625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c647ddeb9c642d7aef7bd1fc99cef3c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.624358 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:00.626079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.626435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.626789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] PATH_INFO: `/v3/projects/6a590dc5d68d41f5982a10ca5a6eb748` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.628683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a590dc5d68d41f5982a10ca5a6eb748)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:00.631484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:00.638529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 932/1868] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:00 2022] DELETE /identity/v3/users/5c647ddeb9c642d7aef7bd1fc99cef3c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:00.639373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:00.640729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.696790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a590dc5d68d41f5982a10ca5a6eb748', 'request_id': 'req-4d751dd0-0380-45fa-b409-f696d8f4c55f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.704764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a590dc5d68d41f5982a10ca5a6eb748', 'request_id': 'req-4d751dd0-0380-45fa-b409-f696d8f4c55f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.712820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a590dc5d68d41f5982a10ca5a6eb748', 'request_id': 'req-4d751dd0-0380-45fa-b409-f696d8f4c55f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.722868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ac3f4-4aaa-407a-94c9-0db30c6cd081 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:00.732340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] The token cache is being invalidate because project 6a590dc5d68d41f5982a10ca5a6eb748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:00.732738 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.733528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-4d751dd0-0380-45fa-b409-f696d8f4c55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:00.735956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 937/1869] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:00 2022] DELETE /identity/v3/projects/6a590dc5d68d41f5982a10ca5a6eb748 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:00.740273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710e95c-9bb3-4b59-ae7b-377534f067d8 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.740273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710e95c-9bb3-4b59-ae7b-377534f067d8 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.740273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710e95c-9bb3-4b59-ae7b-377534f067d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.779423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-d710e95c-9bb3-4b59-ae7b-377534f067d8 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:00.781045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d710e95c-9bb3-4b59-ae7b-377534f067d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.819642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1104f2-1ca8-4822-960d-e9c3b78cde51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:00.821335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1104f2-1ca8-4822-960d-e9c3b78cde51 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.821906 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1104f2-1ca8-4822-960d-e9c3b78cde51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.822377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1104f2-1ca8-4822-960d-e9c3b78cde51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.823433 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1104f2-1ca8-4822-960d-e9c3b78cde51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:00.825463 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1104f2-1ca8-4822-960d-e9c3b78cde51 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:00.840282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 933/1870] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:00.842718 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5041-c841-47db-b5c8-bc820160d553 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.843315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5041-c841-47db-b5c8-bc820160d553 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.843985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5041-c841-47db-b5c8-bc820160d553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.869493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4ccd5041-c841-47db-b5c8-bc820160d553 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:00.871641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4ccd5041-c841-47db-b5c8-bc820160d553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.874679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 938/1871] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:00.875609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710e95c-9bb3-4b59-ae7b-377534f067d8 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:00.876973 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d710e95c-9bb3-4b59-ae7b-377534f067d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:00.885400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 934/1872] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:00.886990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5041-c841-47db-b5c8-bc820160d553 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:00.897403 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710e95c-9bb3-4b59-ae7b-377534f067d8 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:00.899126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5041-c841-47db-b5c8-bc820160d553 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:00.903292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855e642-20de-49d3-8c91-95ccf860329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:00.904003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855e642-20de-49d3-8c91-95ccf860329a admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.904292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855e642-20de-49d3-8c91-95ccf860329a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.904687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855e642-20de-49d3-8c91-95ccf860329a admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.904977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5273fc-870e-4c0d-9fba-8bf94362d205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:00.905583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855e642-20de-49d3-8c91-95ccf860329a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:00.905829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5273fc-870e-4c0d-9fba-8bf94362d205 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.906206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5273fc-870e-4c0d-9fba-8bf94362d205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.906571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5273fc-870e-4c0d-9fba-8bf94362d205 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.907837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855e642-20de-49d3-8c91-95ccf860329a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:00.907978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5273fc-870e-4c0d-9fba-8bf94362d205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:00.908336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5273fc-870e-4c0d-9fba-8bf94362d205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:00.911808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:00.912791 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5273fc-870e-4c0d-9fba-8bf94362d205 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:00.928033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 939/1873] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:00.929727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc1b83-b7db-4213-9dba-356ffa9161bf None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.930298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc1b83-b7db-4213-9dba-356ffa9161bf None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.930802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc1b83-b7db-4213-9dba-356ffa9161bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.934665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 935/1874] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:00.935751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5273fc-870e-4c0d-9fba-8bf94362d205 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:00.949513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5273fc-870e-4c0d-9fba-8bf94362d205 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:00.954653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc1dc5-fa35-446c-a99b-b6c4f4ecd506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:00.955706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1dc5-fa35-446c-a99b-b6c4f4ecd506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:00.955991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1dc5-fa35-446c-a99b-b6c4f4ecd506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:00.956452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1dc5-fa35-446c-a99b-b6c4f4ecd506 admin admin] PATH_INFO: `/v3/projects/05c72e71d89d4f68b7699689921ea528/users/65585cba1f3245d3b4f72ade7be760ef/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:00.960799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:00.984451 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc1dc5-fa35-446c-a99b-b6c4f4ecd506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c72e71d89d4f68b7699689921ea528, user_id=65585cba1f3245d3b4f72ade7be760ef, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:00.986913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc1dc5-fa35-446c-a99b-b6c4f4ecd506 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.003787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e4cc1b83-b7db-4213-9dba-356ffa9161bf None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:01.007486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e4cc1b83-b7db-4213-9dba-356ffa9161bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:01.023933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 936/1875] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:00 2022] PUT /identity/v3/projects/05c72e71d89d4f68b7699689921ea528/users/65585cba1f3245d3b4f72ade7be760ef/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:01.023933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc1dc5-fa35-446c-a99b-b6c4f4ecd506 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.129683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc1dc5-fa35-446c-a99b-b6c4f4ecd506 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.142648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7e892-5bca-46a3-9580-eb674a0924d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.143351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7e892-5bca-46a3-9580-eb674a0924d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.143636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7e892-5bca-46a3-9580-eb674a0924d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.143915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7e892-5bca-46a3-9580-eb674a0924d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.144654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7e892-5bca-46a3-9580-eb674a0924d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.146619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7e892-5bca-46a3-9580-eb674a0924d5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.151308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 940/1876] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:01.153842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f790e-6af1-45fa-a59a-323297dfe6a3 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.154397 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f790e-6af1-45fa-a59a-323297dfe6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.154694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f790e-6af1-45fa-a59a-323297dfe6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.191879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e54f790e-6af1-45fa-a59a-323297dfe6a3 None None] MFA Rules not processed for user `65585cba1f3245d3b4f72ade7be760ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:01.192970 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 937/1877] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:01 2022] POST /identity/v3/projects => generated 369 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:01.193633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e54f790e-6af1-45fa-a59a-323297dfe6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:01.194444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7e892-5bca-46a3-9580-eb674a0924d5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.197857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-08c7e892-5bca-46a3-9580-eb674a0924d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:01.293192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7e892-5bca-46a3-9580-eb674a0924d5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.295203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 941/1878] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:01.297279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f790e-6af1-45fa-a59a-323297dfe6a3 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.300546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.301665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.302190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.302966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] PATH_INFO: `/v3/projects/1478a92784584041930bd14e6abab415` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.305117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1478a92784584041930bd14e6abab415)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.308194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.310453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f790e-6af1-45fa-a59a-323297dfe6a3 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.315926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74668f6f-791f-45e7-aee8-60dc28f713f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.316927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74668f6f-791f-45e7-aee8-60dc28f713f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.319640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74668f6f-791f-45e7-aee8-60dc28f713f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.320004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74668f6f-791f-45e7-aee8-60dc28f713f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.320859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74668f6f-791f-45e7-aee8-60dc28f713f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:01.321319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74668f6f-791f-45e7-aee8-60dc28f713f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.324609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74668f6f-791f-45e7-aee8-60dc28f713f0 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.338551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 942/1879] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:01.339545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74668f6f-791f-45e7-aee8-60dc28f713f0 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.352278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74668f6f-791f-45e7-aee8-60dc28f713f0 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.357780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377678a7-af2d-4b15-acc9-47e9ea037f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.358458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377678a7-af2d-4b15-acc9-47e9ea037f56 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.358745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377678a7-af2d-4b15-acc9-47e9ea037f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.359022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377678a7-af2d-4b15-acc9-47e9ea037f56 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.359805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377678a7-af2d-4b15-acc9-47e9ea037f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.361877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377678a7-af2d-4b15-acc9-47e9ea037f56 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.377769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1478a92784584041930bd14e6abab415', 'request_id': 'req-c7d2c712-55de-447e-be0a-80655db185a6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:01.386227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1478a92784584041930bd14e6abab415', 'request_id': 'req-c7d2c712-55de-447e-be0a-80655db185a6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:01.391965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 943/1880] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:01 2022] POST /identity/v3/users => generated 285 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:01.391965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377678a7-af2d-4b15-acc9-47e9ea037f56 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.407935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377678a7-af2d-4b15-acc9-47e9ea037f56 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.414814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1478a92784584041930bd14e6abab415', 'request_id': 'req-c7d2c712-55de-447e-be0a-80655db185a6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:01.440070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:01.451009 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] The token cache is being invalidate because project 1478a92784584041930bd14e6abab415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:01.451304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:01.451998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:01.460121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 938/1881] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:01 2022] DELETE /identity/v3/projects/1478a92784584041930bd14e6abab415 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:01.461088 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.462581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:01.472273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:01.484500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813554f-2886-4fc8-a7a4-52c6fbde39c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.485302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813554f-2886-4fc8-a7a4-52c6fbde39c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.485725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813554f-2886-4fc8-a7a4-52c6fbde39c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.486536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813554f-2886-4fc8-a7a4-52c6fbde39c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.487428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813554f-2886-4fc8-a7a4-52c6fbde39c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.491388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813554f-2886-4fc8-a7a4-52c6fbde39c1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.528311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 944/1882] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:01 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:01.528311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813554f-2886-4fc8-a7a4-52c6fbde39c1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.528311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0813554f-2886-4fc8-a7a4-52c6fbde39c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:01.550403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2c712-55de-447e-be0a-80655db185a6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.606662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813554f-2886-4fc8-a7a4-52c6fbde39c1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.623977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d895fa6-86b1-4e1b-9a13-ee4172d40dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.624888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d895fa6-86b1-4e1b-9a13-ee4172d40dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.625096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d895fa6-86b1-4e1b-9a13-ee4172d40dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.625610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d895fa6-86b1-4e1b-9a13-ee4172d40dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.626371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d895fa6-86b1-4e1b-9a13-ee4172d40dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.628450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d895fa6-86b1-4e1b-9a13-ee4172d40dd3 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.640692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 945/1883] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:01.645941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1610160-5596-4058-af32-ec32b95962de None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.646253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1610160-5596-4058-af32-ec32b95962de None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.646536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1610160-5596-4058-af32-ec32b95962de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.663343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe7967-f6e9-4c5d-b79d-1f4ab2848e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.667806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe7967-f6e9-4c5d-b79d-1f4ab2848e5b admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.668908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe7967-f6e9-4c5d-b79d-1f4ab2848e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.669290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-a1610160-5596-4058-af32-ec32b95962de None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:01.669708 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe7967-f6e9-4c5d-b79d-1f4ab2848e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.670729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a1610160-5596-4058-af32-ec32b95962de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:01.670859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe7967-f6e9-4c5d-b79d-1f4ab2848e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.674281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe7967-f6e9-4c5d-b79d-1f4ab2848e5b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.681580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 946/1884] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:01.682628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1610160-5596-4058-af32-ec32b95962de None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.684104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a1610160-5596-4058-af32-ec32b95962de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:01.700190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1610160-5596-4058-af32-ec32b95962de None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.712754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdb203-97c8-4379-989d-385a848741cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.716989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 939/1885] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:01 2022] POST /identity/v3/projects => generated 363 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:01.717297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdb203-97c8-4379-989d-385a848741cb admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.717297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdb203-97c8-4379-989d-385a848741cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.717297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdb203-97c8-4379-989d-385a848741cb admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.717991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe7967-f6e9-4c5d-b79d-1f4ab2848e5b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.718073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdb203-97c8-4379-989d-385a848741cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.718959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdb203-97c8-4379-989d-385a848741cb admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.758445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 947/1886] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:01 2022] POST /identity/v3/users => generated 297 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:01.758861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3049c7-c95e-40d7-a7de-358d1538fc0f None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.759119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3049c7-c95e-40d7-a7de-358d1538fc0f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.759449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3049c7-c95e-40d7-a7de-358d1538fc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.791371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-df3049c7-c95e-40d7-a7de-358d1538fc0f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:01.792825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-df3049c7-c95e-40d7-a7de-358d1538fc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:01.805909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe7967-f6e9-4c5d-b79d-1f4ab2848e5b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.811058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da27509-877c-4471-9ec6-bc9aecc82de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.811964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27509-877c-4471-9ec6-bc9aecc82de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.812504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27509-877c-4471-9ec6-bc9aecc82de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.812856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27509-877c-4471-9ec6-bc9aecc82de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.813666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da27509-877c-4471-9ec6-bc9aecc82de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.815537 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da27509-877c-4471-9ec6-bc9aecc82de7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.828632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 940/1887] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:01.829539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da27509-877c-4471-9ec6-bc9aecc82de7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.848836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da27509-877c-4471-9ec6-bc9aecc82de7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.855873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a60f71-179d-4439-a5ae-a69a71ed6ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.856784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a60f71-179d-4439-a5ae-a69a71ed6ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.857241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a60f71-179d-4439-a5ae-a69a71ed6ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.857593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a60f71-179d-4439-a5ae-a69a71ed6ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.858658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a60f71-179d-4439-a5ae-a69a71ed6ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.860856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a60f71-179d-4439-a5ae-a69a71ed6ee9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.867846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 948/1888] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:01.868880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3049c7-c95e-40d7-a7de-358d1538fc0f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.879210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3049c7-c95e-40d7-a7de-358d1538fc0f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.887473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a9a8a-3a19-4f6c-bdc7-b233034ba2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.888430 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9a8a-3a19-4f6c-bdc7-b233034ba2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.890121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9a8a-3a19-4f6c-bdc7-b233034ba2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.890508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9a8a-3a19-4f6c-bdc7-b233034ba2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.891688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a9a8a-3a19-4f6c-bdc7-b233034ba2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.894599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a9a8a-3a19-4f6c-bdc7-b233034ba2c5 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.900807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 941/1889] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:01 2022] POST /identity/v3/users => generated 282 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:01.900807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a60f71-179d-4439-a5ae-a69a71ed6ee9 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.922800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 949/1890] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:01.924325 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a9a8a-3a19-4f6c-bdc7-b233034ba2c5 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:01.930626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a60f71-179d-4439-a5ae-a69a71ed6ee9 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.944048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c656e-e5aa-4bb0-ae65-db6b8d3d502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.945241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c656e-e5aa-4bb0-ae65-db6b8d3d502e admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.945631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c656e-e5aa-4bb0-ae65-db6b8d3d502e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.946009 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c656e-e5aa-4bb0-ae65-db6b8d3d502e admin admin] PATH_INFO: `/v3/projects/c5488c571c4a4fe0a7127004a2bb7b35/users/b9b04954e4d64ab6859ef8508beb855f/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.949582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a9a8a-3a19-4f6c-bdc7-b233034ba2c5 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:01.962609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:01.963624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fec3e3-230a-4179-90f6-4c393c295274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:01.964847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fec3e3-230a-4179-90f6-4c393c295274 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:01.965360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fec3e3-230a-4179-90f6-4c393c295274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:01.965736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fec3e3-230a-4179-90f6-4c393c295274 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:01.966617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fec3e3-230a-4179-90f6-4c393c295274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:01.967030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fec3e3-230a-4179-90f6-4c393c295274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.968846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fec3e3-230a-4179-90f6-4c393c295274 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:01.981548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c656e-e5aa-4bb0-ae65-db6b8d3d502e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5488c571c4a4fe0a7127004a2bb7b35, user_id=b9b04954e4d64ab6859ef8508beb855f, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:01.986363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c656e-e5aa-4bb0-ae65-db6b8d3d502e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.000713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 950/1891] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:02.002284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fec3e3-230a-4179-90f6-4c393c295274 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.018446 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 942/1892] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:01 2022] PUT /identity/v3/projects/c5488c571c4a4fe0a7127004a2bb7b35/users/b9b04954e4d64ab6859ef8508beb855f/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:02.019138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c656e-e5aa-4bb0-ae65-db6b8d3d502e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.023997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fec3e3-230a-4179-90f6-4c393c295274 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.071402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:02.115494 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d1155-b86e-4d2f-8713-6d97434ebdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.116497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d1155-b86e-4d2f-8713-6d97434ebdcf admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.116926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d1155-b86e-4d2f-8713-6d97434ebdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.117346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d1155-b86e-4d2f-8713-6d97434ebdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.118291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d1155-b86e-4d2f-8713-6d97434ebdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.120471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d1155-b86e-4d2f-8713-6d97434ebdcf admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.138697 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 951/1893] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:02.138697 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d1155-b86e-4d2f-8713-6d97434ebdcf admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.151722 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c656e-e5aa-4bb0-ae65-db6b8d3d502e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.152782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d1155-b86e-4d2f-8713-6d97434ebdcf admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.156918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6c48-e6dd-4296-bda4-a4d2b575bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.157884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6c48-e6dd-4296-bda4-a4d2b575bef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.158167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6c48-e6dd-4296-bda4-a4d2b575bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.158264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd9868-f355-4155-bc19-0bf2ae9832db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.158437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6c48-e6dd-4296-bda4-a4d2b575bef4 admin admin] PATH_INFO: `/v3/projects/31ef94d2667d42d5b0b1eca8084ecc15/users/57f404b1ea3f417881fa0a266c1aba35/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.158878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd9868-f355-4155-bc19-0bf2ae9832db admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.159166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd9868-f355-4155-bc19-0bf2ae9832db admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.159527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd9868-f355-4155-bc19-0bf2ae9832db admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.160262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd9868-f355-4155-bc19-0bf2ae9832db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.162841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd9868-f355-4155-bc19-0bf2ae9832db admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.177968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e6c48-e6dd-4296-bda4-a4d2b575bef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ef94d2667d42d5b0b1eca8084ecc15, user_id=57f404b1ea3f417881fa0a266c1aba35, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.180693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e6c48-e6dd-4296-bda4-a4d2b575bef4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.195737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 952/1894] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:02 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:02.197115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a850c2-c609-4d40-ac8b-35dfac62abd0 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.197410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a850c2-c609-4d40-ac8b-35dfac62abd0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.198318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a850c2-c609-4d40-ac8b-35dfac62abd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.202533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 943/1895] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:02 2022] PUT /identity/v3/projects/31ef94d2667d42d5b0b1eca8084ecc15/users/57f404b1ea3f417881fa0a266c1aba35/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:02.203455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6c48-e6dd-4296-bda4-a4d2b575bef4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.228904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b0a850c2-c609-4d40-ac8b-35dfac62abd0 None None] MFA Rules not processed for user `b9b04954e4d64ab6859ef8508beb855f`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:02.230557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b0a850c2-c609-4d40-ac8b-35dfac62abd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:02.277091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6c48-e6dd-4296-bda4-a4d2b575bef4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.314872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 953/1896] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:02.316081 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a850c2-c609-4d40-ac8b-35dfac62abd0 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.359061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a6d4f-f308-47b9-9ed4-1efc4e38bfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.360344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a6d4f-f308-47b9-9ed4-1efc4e38bfe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.360898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a6d4f-f308-47b9-9ed4-1efc4e38bfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.361404 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a6d4f-f308-47b9-9ed4-1efc4e38bfe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.362633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a6d4f-f308-47b9-9ed4-1efc4e38bfe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.365228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a6d4f-f308-47b9-9ed4-1efc4e38bfe5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.376145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 944/1897] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:02.377143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a6d4f-f308-47b9-9ed4-1efc4e38bfe5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.387564 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a6d4f-f308-47b9-9ed4-1efc4e38bfe5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.392306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1945f37-6236-4bc7-828d-28974f5330f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.393078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1945f37-6236-4bc7-828d-28974f5330f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.393689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1945f37-6236-4bc7-828d-28974f5330f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.393689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1945f37-6236-4bc7-828d-28974f5330f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.394548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1945f37-6236-4bc7-828d-28974f5330f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.396429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1945f37-6236-4bc7-828d-28974f5330f4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.399330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a850c2-c609-4d40-ac8b-35dfac62abd0 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.404058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933da07-3f77-45c9-94b4-c1a86f8cb98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.404896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933da07-3f77-45c9-94b4-c1a86f8cb98b admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.405097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933da07-3f77-45c9-94b4-c1a86f8cb98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.405347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933da07-3f77-45c9-94b4-c1a86f8cb98b admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.406049 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933da07-3f77-45c9-94b4-c1a86f8cb98b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.408454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933da07-3f77-45c9-94b4-c1a86f8cb98b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.408570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 945/1898] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:02.414767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1945f37-6236-4bc7-828d-28974f5330f4 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.420146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:02.426274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1945f37-6236-4bc7-828d-28974f5330f4 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.434858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 954/1899] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:02 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:02.434858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933da07-3f77-45c9-94b4-c1a86f8cb98b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.436695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ea7c4-694b-4a81-9cf9-6db1e5e82e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.437954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ea7c4-694b-4a81-9cf9-6db1e5e82e24 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.438377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ea7c4-694b-4a81-9cf9-6db1e5e82e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.438751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ea7c4-694b-4a81-9cf9-6db1e5e82e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.440223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ea7c4-694b-4a81-9cf9-6db1e5e82e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.443519 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ea7c4-694b-4a81-9cf9-6db1e5e82e24 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.444160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933da07-3f77-45c9-94b4-c1a86f8cb98b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.452198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0332bf-6087-4416-a78e-3075b9570eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.452907 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0332bf-6087-4416-a78e-3075b9570eed admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.453200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0332bf-6087-4416-a78e-3075b9570eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.454743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0332bf-6087-4416-a78e-3075b9570eed admin admin] PATH_INFO: `/v3/projects/46a6939ef23042c6abc2a3683ed9b11b/users/87ba96a5aae6459999dadaf7f20b8cb5/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.488031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 946/1900] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:02 2022] POST /identity/v3/projects => generated 369 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:02.488031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ea7c4-694b-4a81-9cf9-6db1e5e82e24 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.492101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0332bf-6087-4416-a78e-3075b9570eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a6939ef23042c6abc2a3683ed9b11b, user_id=87ba96a5aae6459999dadaf7f20b8cb5, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.494887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0332bf-6087-4416-a78e-3075b9570eed admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.505051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:02.533416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 955/1901] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:02 2022] PUT /identity/v3/projects/46a6939ef23042c6abc2a3683ed9b11b/users/87ba96a5aae6459999dadaf7f20b8cb5/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:02.533416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0332bf-6087-4416-a78e-3075b9570eed admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.570329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ea7c4-694b-4a81-9cf9-6db1e5e82e24 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.608771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0332bf-6087-4416-a78e-3075b9570eed admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.613551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb8d93-c266-406e-8110-231b103abc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.614580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8d93-c266-406e-8110-231b103abc3c admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.615126 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8d93-c266-406e-8110-231b103abc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.615392 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb8d93-c266-406e-8110-231b103abc3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.616305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb8d93-c266-406e-8110-231b103abc3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.617951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb8d93-c266-406e-8110-231b103abc3c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.631156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 956/1902] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:02.631156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb8d93-c266-406e-8110-231b103abc3c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.640409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb8d93-c266-406e-8110-231b103abc3c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.648643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc00c7-85a0-4a96-8800-405c20e9eedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.649316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc00c7-85a0-4a96-8800-405c20e9eedf admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.649603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc00c7-85a0-4a96-8800-405c20e9eedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.649878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc00c7-85a0-4a96-8800-405c20e9eedf admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.651075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc00c7-85a0-4a96-8800-405c20e9eedf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.653454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6aecc2-d51a-439f-a192-b727df1019e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.656335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc00c7-85a0-4a96-8800-405c20e9eedf admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.660533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aecc2-d51a-439f-a192-b727df1019e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.661421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aecc2-d51a-439f-a192-b727df1019e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.661715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aecc2-d51a-439f-a192-b727df1019e7 admin admin] PATH_INFO: `/v3/projects/31ef94d2667d42d5b0b1eca8084ecc15/users/57f404b1ea3f417881fa0a266c1aba35/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.665453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6aecc2-d51a-439f-a192-b727df1019e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ef94d2667d42d5b0b1eca8084ecc15, user_id=57f404b1ea3f417881fa0a266c1aba35, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.668965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6aecc2-d51a-439f-a192-b727df1019e7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.689454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 947/1903] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:02 2022] PUT /identity/v3/projects/31ef94d2667d42d5b0b1eca8084ecc15/users/57f404b1ea3f417881fa0a266c1aba35/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:02.707287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c898c-0387-4c58-8c83-f2f7cf696b54 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.707287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c898c-0387-4c58-8c83-f2f7cf696b54 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.707287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c898c-0387-4c58-8c83-f2f7cf696b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.707696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 957/1904] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:02 2022] POST /identity/v3/users => generated 291 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:02.707696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc00c7-85a0-4a96-8800-405c20e9eedf admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.717511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc00c7-85a0-4a96-8800-405c20e9eedf admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.735728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b87c898c-0387-4c58-8c83-f2f7cf696b54 None None] MFA Rules not processed for user `87ba96a5aae6459999dadaf7f20b8cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:02.738468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b87c898c-0387-4c58-8c83-f2f7cf696b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:02.853379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 948/1905] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:02.856965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5db86a-e54d-4061-a425-8170a257a373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.856965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5db86a-e54d-4061-a425-8170a257a373 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.856965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5db86a-e54d-4061-a425-8170a257a373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.856965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5db86a-e54d-4061-a425-8170a257a373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.857566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5db86a-e54d-4061-a425-8170a257a373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.858543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c898c-0387-4c58-8c83-f2f7cf696b54 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.859655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5db86a-e54d-4061-a425-8170a257a373 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.873498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c898c-0387-4c58-8c83-f2f7cf696b54 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.875051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 958/1906] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:02.876380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab588a-fd2b-485c-a536-5258863f3419 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.876703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab588a-fd2b-485c-a536-5258863f3419 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.876983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab588a-fd2b-485c-a536-5258863f3419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.887542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec26876-587c-41b2-b729-86895f42396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:02.888655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec26876-587c-41b2-b729-86895f42396f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:02.889295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec26876-587c-41b2-b729-86895f42396f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:02.890127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec26876-587c-41b2-b729-86895f42396f admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:02.891328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec26876-587c-41b2-b729-86895f42396f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:02.893092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec26876-587c-41b2-b729-86895f42396f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:02.901497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:02.914065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 949/1907] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:02.917383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec26876-587c-41b2-b729-86895f42396f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:02.953590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec26876-587c-41b2-b729-86895f42396f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:02.956258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-00ab588a-fd2b-485c-a536-5258863f3419 None None] MFA Rules not processed for user `57f404b1ea3f417881fa0a266c1aba35`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:02.958479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-00ab588a-fd2b-485c-a536-5258863f3419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:02.999001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71420cc-6b05-42f2-a14d-e87167ff5af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.000847 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71420cc-6b05-42f2-a14d-e87167ff5af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.001449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71420cc-6b05-42f2-a14d-e87167ff5af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.002027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71420cc-6b05-42f2-a14d-e87167ff5af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.003118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71420cc-6b05-42f2-a14d-e87167ff5af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.005909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71420cc-6b05-42f2-a14d-e87167ff5af2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.045940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 950/1908] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:02 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:03.047416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71420cc-6b05-42f2-a14d-e87167ff5af2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.076126 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:03.096433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 959/1909] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:03.097586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab588a-fd2b-485c-a536-5258863f3419 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.120874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab588a-fd2b-485c-a536-5258863f3419 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.143210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71420cc-6b05-42f2-a14d-e87167ff5af2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.151245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a0838-7c83-408c-9c09-02b228a52783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.152614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a0838-7c83-408c-9c09-02b228a52783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.153503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a0838-7c83-408c-9c09-02b228a52783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.153866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a0838-7c83-408c-9c09-02b228a52783 admin admin] PATH_INFO: `/v3/projects/2fbb5dd6302f461b862ca3bacf421fd1/users/04771d8ce78b4157bc382242c382714b/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.176925 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a0838-7c83-408c-9c09-02b228a52783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fbb5dd6302f461b862ca3bacf421fd1, user_id=04771d8ce78b4157bc382242c382714b, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.180758 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a0838-7c83-408c-9c09-02b228a52783 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.205686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 951/1910] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:03 2022] PUT /identity/v3/projects/2fbb5dd6302f461b862ca3bacf421fd1/users/04771d8ce78b4157bc382242c382714b/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:03.206575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a0838-7c83-408c-9c09-02b228a52783 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.215130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78978582-8c95-43e3-8e88-268dbec94456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.216103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78978582-8c95-43e3-8e88-268dbec94456 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.216433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78978582-8c95-43e3-8e88-268dbec94456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.216741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78978582-8c95-43e3-8e88-268dbec94456 admin admin] PATH_INFO: `/v3/projects/0baadba2aaa04a23b6152813d58f304a/users/554dd8bbf4454009a233e5f56ddff01c/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.236116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78978582-8c95-43e3-8e88-268dbec94456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0baadba2aaa04a23b6152813d58f304a, user_id=554dd8bbf4454009a233e5f56ddff01c, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.240035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78978582-8c95-43e3-8e88-268dbec94456 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.269153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 960/1911] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:03 2022] PUT /identity/v3/projects/0baadba2aaa04a23b6152813d58f304a/users/554dd8bbf4454009a233e5f56ddff01c/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:03.273166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78978582-8c95-43e3-8e88-268dbec94456 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.274940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-78978582-8c95-43e3-8e88-268dbec94456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:03.299557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a0838-7c83-408c-9c09-02b228a52783 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.383266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93484476-2e21-443e-bf9f-9a48adbf3c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.384676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93484476-2e21-443e-bf9f-9a48adbf3c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.385133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93484476-2e21-443e-bf9f-9a48adbf3c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.385550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93484476-2e21-443e-bf9f-9a48adbf3c4b admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.386397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93484476-2e21-443e-bf9f-9a48adbf3c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.388632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93484476-2e21-443e-bf9f-9a48adbf3c4b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.400898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78978582-8c95-43e3-8e88-268dbec94456 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.405802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf62f89-61c0-439a-a4e8-222eb50e893a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.406615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62f89-61c0-439a-a4e8-222eb50e893a service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.407123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62f89-61c0-439a-a4e8-222eb50e893a service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.407502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62f89-61c0-439a-a4e8-222eb50e893a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.408488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:03.419525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 952/1912] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:03 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:03.421148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1586be-a795-43bd-984b-df3033f7d843 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.422097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1586be-a795-43bd-984b-df3033f7d843 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.422441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1586be-a795-43bd-984b-df3033f7d843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.452490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-db1586be-a795-43bd-984b-df3033f7d843 None None] MFA Rules not processed for user `04771d8ce78b4157bc382242c382714b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:03.453875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-db1586be-a795-43bd-984b-df3033f7d843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:03.514960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf62f89-61c0-439a-a4e8-222eb50e893a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.518265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf62f89-61c0-439a-a4e8-222eb50e893a service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.539744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 961/1913] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:03.540717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf62f89-61c0-439a-a4e8-222eb50e893a service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.544506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 953/1914] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:03.544506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1586be-a795-43bd-984b-df3033f7d843 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.555599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1586be-a795-43bd-984b-df3033f7d843 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.558747 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf62f89-61c0-439a-a4e8-222eb50e893a service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.560684 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a77c1-cab1-415d-a188-63038af60833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.561470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a77c1-cab1-415d-a188-63038af60833 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.561857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a77c1-cab1-415d-a188-63038af60833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.562235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a77c1-cab1-415d-a188-63038af60833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.563125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a77c1-cab1-415d-a188-63038af60833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.564472 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e950a-633d-4642-aff2-75a18a5bf2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.565352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e950a-633d-4642-aff2-75a18a5bf2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.565472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a77c1-cab1-415d-a188-63038af60833 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.565581 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e950a-633d-4642-aff2-75a18a5bf2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.565581 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e950a-633d-4642-aff2-75a18a5bf2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.566363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e950a-633d-4642-aff2-75a18a5bf2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.569203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e950a-633d-4642-aff2-75a18a5bf2d7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.581805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 962/1915] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:03.583419 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e950a-633d-4642-aff2-75a18a5bf2d7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.583974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 954/1916] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:03.590574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a77c1-cab1-415d-a188-63038af60833 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.590938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-688e950a-633d-4642-aff2-75a18a5bf2d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:03.605611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e950a-633d-4642-aff2-75a18a5bf2d7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.605969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a77c1-cab1-415d-a188-63038af60833 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.610479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50a93f-0ebb-4caf-9ca9-d100fbcfd386 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.611725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50a93f-0ebb-4caf-9ca9-d100fbcfd386 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.612146 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50a93f-0ebb-4caf-9ca9-d100fbcfd386 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.612464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50a93f-0ebb-4caf-9ca9-d100fbcfd386 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.615266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b2e00-aecf-4273-8a1a-dbadbc988378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.617489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b2e00-aecf-4273-8a1a-dbadbc988378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.617995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b2e00-aecf-4273-8a1a-dbadbc988378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.618488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b2e00-aecf-4273-8a1a-dbadbc988378 admin admin] PATH_INFO: `/v3/projects/ff15977e143b4c3ca408d57558385175/users/d5ebcf0e844b480b9c0038bcf4ae12a3/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.626769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50a93f-0ebb-4caf-9ca9-d100fbcfd386 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.628650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50a93f-0ebb-4caf-9ca9-d100fbcfd386 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.640829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b2e00-aecf-4273-8a1a-dbadbc988378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff15977e143b4c3ca408d57558385175, user_id=d5ebcf0e844b480b9c0038bcf4ae12a3, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.641145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 963/1917] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:03.642067 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50a93f-0ebb-4caf-9ca9-d100fbcfd386 service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.646181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b2e00-aecf-4273-8a1a-dbadbc988378 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.665680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50a93f-0ebb-4caf-9ca9-d100fbcfd386 service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.674739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 955/1918] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:03 2022] PUT /identity/v3/projects/ff15977e143b4c3ca408d57558385175/users/d5ebcf0e844b480b9c0038bcf4ae12a3/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:03.675607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b2e00-aecf-4273-8a1a-dbadbc988378 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.678089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:03.764653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40d16c-3199-44fd-a53b-1292870a27fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.765560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40d16c-3199-44fd-a53b-1292870a27fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.765843 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40d16c-3199-44fd-a53b-1292870a27fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.766125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40d16c-3199-44fd-a53b-1292870a27fb admin admin] PATH_INFO: `/v3/projects/0baadba2aaa04a23b6152813d58f304a/users/554dd8bbf4454009a233e5f56ddff01c/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.768414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40d16c-3199-44fd-a53b-1292870a27fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0baadba2aaa04a23b6152813d58f304a, user_id=554dd8bbf4454009a233e5f56ddff01c, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.768780 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b2e00-aecf-4273-8a1a-dbadbc988378 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.772143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40d16c-3199-44fd-a53b-1292870a27fb admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.776199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.781215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.781215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.781215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] PATH_INFO: `/v3/users/04771d8ce78b4157bc382242c382714b` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.781215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] RBAC: Authorizing `identity:delete_user(user_id=04771d8ce78b4157bc382242c382714b)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.781775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.797289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 964/1919] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:03 2022] PUT /identity/v3/projects/0baadba2aaa04a23b6152813d58f304a/users/554dd8bbf4454009a233e5f56ddff01c/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:03.798383 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40d16c-3199-44fd-a53b-1292870a27fb admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:03.895688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04771d8ce78b4157bc382242c382714b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:03.896766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04771d8ce78b4157bc382242c382714b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:03.919924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:03.928854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04771d8ce78b4157bc382242c382714b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:03.932826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40d16c-3199-44fd-a53b-1292870a27fb admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:03.938047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04771d8ce78b4157bc382242c382714b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:03.946395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-37f7961c-00ba-4c94-902a-85bdcf1b501e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04771d8ce78b4157bc382242c382714b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:03.959895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3141d-ff89-4d23-8b50-f7fde6f474b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:03.962913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3141d-ff89-4d23-8b50-f7fde6f474b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.963206 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3141d-ff89-4d23-8b50-f7fde6f474b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.963488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3141d-ff89-4d23-8b50-f7fde6f474b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:03.965196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3141d-ff89-4d23-8b50-f7fde6f474b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:03.968809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3141d-ff89-4d23-8b50-f7fde6f474b1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:03.971855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 956/1920] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:03 2022] DELETE /identity/v3/users/04771d8ce78b4157bc382242c382714b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:03.977291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e3b36-b1ea-4e10-a9a0-917d7d63fa03 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:03.977657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e3b36-b1ea-4e10-a9a0-917d7d63fa03 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:03.978197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e3b36-b1ea-4e10-a9a0-917d7d63fa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:04.011597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 965/1921] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:04.016291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bec0df-7f5d-462b-86a0-5b78f809eb36 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:04.016614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bec0df-7f5d-462b-86a0-5b78f809eb36 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:04.016895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bec0df-7f5d-462b-86a0-5b78f809eb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:04.038245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fd1e3b36-b1ea-4e10-a9a0-917d7d63fa03 None None] MFA Rules not processed for user `554dd8bbf4454009a233e5f56ddff01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:04.039519 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fd1e3b36-b1ea-4e10-a9a0-917d7d63fa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.086828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-95bec0df-7f5d-462b-86a0-5b78f809eb36 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:04.088192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-95bec0df-7f5d-462b-86a0-5b78f809eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.197658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 957/1922] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:04.198460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e3b36-b1ea-4e10-a9a0-917d7d63fa03 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:04.204862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fd1e3b36-b1ea-4e10-a9a0-917d7d63fa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.240323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 966/1923] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:04.241298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bec0df-7f5d-462b-86a0-5b78f809eb36 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:04.242623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-95bec0df-7f5d-462b-86a0-5b78f809eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.265556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bec0df-7f5d-462b-86a0-5b78f809eb36 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:04.270816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:04.271836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:04.272117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:04.272542 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] PATH_INFO: `/v3/users/65585cba1f3245d3b4f72ade7be760ef` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:04.281459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] RBAC: Authorizing `identity:delete_user(user_id=65585cba1f3245d3b4f72ade7be760ef)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:04.285367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:04.344309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e3b36-b1ea-4e10-a9a0-917d7d63fa03 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:04.349951 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401853e-41a2-4432-931c-23a7d8aeeb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:04.352309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401853e-41a2-4432-931c-23a7d8aeeb1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:04.352309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401853e-41a2-4432-931c-23a7d8aeeb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:04.352309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401853e-41a2-4432-931c-23a7d8aeeb1f admin admin] PATH_INFO: `/v3/projects/ff15977e143b4c3ca408d57558385175/users/d5ebcf0e844b480b9c0038bcf4ae12a3/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:04.354079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401853e-41a2-4432-931c-23a7d8aeeb1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff15977e143b4c3ca408d57558385175, user_id=d5ebcf0e844b480b9c0038bcf4ae12a3, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:04.354791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:04.357916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401853e-41a2-4432-931c-23a7d8aeeb1f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:04.367915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65585cba1f3245d3b4f72ade7be760ef'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:04.368826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65585cba1f3245d3b4f72ade7be760ef'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:04.371653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 958/1924] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:04 2022] PUT /identity/v3/projects/ff15977e143b4c3ca408d57558385175/users/d5ebcf0e844b480b9c0038bcf4ae12a3/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:04.377054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65585cba1f3245d3b4f72ade7be760ef'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:04.380942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65585cba1f3245d3b4f72ade7be760ef'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:04.382471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401853e-41a2-4432-931c-23a7d8aeeb1f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:04.384329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7401853e-41a2-4432-931c-23a7d8aeeb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.396825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65585cba1f3245d3b4f72ade7be760ef'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:04.415488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 967/1925] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:04 2022] DELETE /identity/v3/users/65585cba1f3245d3b4f72ade7be760ef => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:04.416580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:04.420550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.456220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401853e-41a2-4432-931c-23a7d8aeeb1f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:04.462057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:04.493345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af59372-f3cc-46a7-9c88-1ffc90e7f55b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:04.497322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730fc83-1417-4669-a169-10a8e60e91ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:04.497971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730fc83-1417-4669-a169-10a8e60e91ea service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:04.498401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730fc83-1417-4669-a169-10a8e60e91ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:04.498687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730fc83-1417-4669-a169-10a8e60e91ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:04.500510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b730fc83-1417-4669-a169-10a8e60e91ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.522206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2e7d9-231e-4908-97e9-05b2c7144215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:04.522882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2e7d9-231e-4908-97e9-05b2c7144215 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:04.523207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2e7d9-231e-4908-97e9-05b2c7144215 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:04.523442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2e7d9-231e-4908-97e9-05b2c7144215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:04.525323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-96d2e7d9-231e-4908-97e9-05b2c7144215 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.577741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730fc83-1417-4669-a169-10a8e60e91ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:04.580176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730fc83-1417-4669-a169-10a8e60e91ea service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:04.595093 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 968/1926] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:04.596796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a931-5b2c-4fc1-b4e5-26808fa964f0 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:04.597092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a931-5b2c-4fc1-b4e5-26808fa964f0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:04.597374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a931-5b2c-4fc1-b4e5-26808fa964f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:04.634323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2e7d9-231e-4908-97e9-05b2c7144215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:04.635830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2e7d9-231e-4908-97e9-05b2c7144215 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:04.639284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-1655a931-5b2c-4fc1-b4e5-26808fa964f0 None None] MFA Rules not processed for user `d5ebcf0e844b480b9c0038bcf4ae12a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:04.641859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1655a931-5b2c-4fc1-b4e5-26808fa964f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.650749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 959/1927] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:04 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:04.668774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d582f0-6900-40e3-b775-9026d45a5ab9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:04.669111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d582f0-6900-40e3-b775-9026d45a5ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:04.669389 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d582f0-6900-40e3-b775-9026d45a5ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:04.711097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-54d582f0-6900-40e3-b775-9026d45a5ab9 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:04.712808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-54d582f0-6900-40e3-b775-9026d45a5ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.751681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 960/1928] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:04.752682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d582f0-6900-40e3-b775-9026d45a5ab9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:04.756673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-54d582f0-6900-40e3-b775-9026d45a5ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:04.759228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:04.788371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d582f0-6900-40e3-b775-9026d45a5ab9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:04.792985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59276871-6628-4527-abca-b8f6f55b3ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:04.793655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59276871-6628-4527-abca-b8f6f55b3ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:04.793947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59276871-6628-4527-abca-b8f6f55b3ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:04.794223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59276871-6628-4527-abca-b8f6f55b3ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:04.809632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59276871-6628-4527-abca-b8f6f55b3ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:04.811362 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59276871-6628-4527-abca-b8f6f55b3ff9 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:04.820678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 969/1929] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:04.823752 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 961/1930] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:04.841980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a931-5b2c-4fc1-b4e5-26808fa964f0 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:04.859646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655a931-5b2c-4fc1-b4e5-26808fa964f0 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:04.868637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163ad5c-c08d-411e-822b-e4bc936ffbc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:04.869707 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163ad5c-c08d-411e-822b-e4bc936ffbc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:04.870994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163ad5c-c08d-411e-822b-e4bc936ffbc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:04.873999 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163ad5c-c08d-411e-822b-e4bc936ffbc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:04.889168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163ad5c-c08d-411e-822b-e4bc936ffbc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:04.891455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163ad5c-c08d-411e-822b-e4bc936ffbc9 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:04.907127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 970/1931] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:05.038471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59276871-6628-4527-abca-b8f6f55b3ff9 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:05.040297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-59276871-6628-4527-abca-b8f6f55b3ff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:05.061263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59276871-6628-4527-abca-b8f6f55b3ff9 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:05.085544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:05.108044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:05.108552 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:05.123396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] PATH_INFO: `/v3/projects/2fbb5dd6302f461b862ca3bacf421fd1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:05.125596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fbb5dd6302f461b862ca3bacf421fd1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:05.128893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:05.135834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163ad5c-c08d-411e-822b-e4bc936ffbc9 service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:05.137307 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c163ad5c-c08d-411e-822b-e4bc936ffbc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:05.154847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163ad5c-c08d-411e-822b-e4bc936ffbc9 service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:05.180112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee6cbf-b140-4840-ae4e-6d9555863556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:05.180816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee6cbf-b140-4840-ae4e-6d9555863556 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:05.181114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee6cbf-b140-4840-ae4e-6d9555863556 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:05.181460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee6cbf-b140-4840-ae4e-6d9555863556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:05.189688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2cee6cbf-b140-4840-ae4e-6d9555863556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:05.235052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fbb5dd6302f461b862ca3bacf421fd1', 'request_id': 'req-48cb8594-c640-4fb0-b749-b839f7967bd1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:05.251801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fbb5dd6302f461b862ca3bacf421fd1', 'request_id': 'req-48cb8594-c640-4fb0-b749-b839f7967bd1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:05.274428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fbb5dd6302f461b862ca3bacf421fd1', 'request_id': 'req-48cb8594-c640-4fb0-b749-b839f7967bd1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:05.310018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] The token cache is being invalidate because project 2fbb5dd6302f461b862ca3bacf421fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:05.310018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:05.310018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-48cb8594-c640-4fb0-b749-b839f7967bd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:05.310018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 962/1932] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:05 2022] DELETE /identity/v3/projects/2fbb5dd6302f461b862ca3bacf421fd1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:05.315836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2849b5-57a8-40e5-bae9-52d6b8440d1d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:05.316177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2849b5-57a8-40e5-bae9-52d6b8440d1d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:05.316580 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2849b5-57a8-40e5-bae9-52d6b8440d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:05.341817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1c2849b5-57a8-40e5-bae9-52d6b8440d1d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:05.343234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1c2849b5-57a8-40e5-bae9-52d6b8440d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:05.371496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:05.375359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee6cbf-b140-4840-ae4e-6d9555863556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:05.388910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee6cbf-b140-4840-ae4e-6d9555863556 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:05.401825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 971/1933] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:05 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:05.455651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 963/1934] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:05.463874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccbbf2-e8aa-473f-a4eb-268e0f9c9dfa None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:05.464072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccbbf2-e8aa-473f-a4eb-268e0f9c9dfa None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:05.465412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccbbf2-e8aa-473f-a4eb-268e0f9c9dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:05.492576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-d8ccbbf2-e8aa-473f-a4eb-268e0f9c9dfa None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:05.494782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d8ccbbf2-e8aa-473f-a4eb-268e0f9c9dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:05.510900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 972/1935] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:05.515392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2849b5-57a8-40e5-bae9-52d6b8440d1d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:05.527281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2849b5-57a8-40e5-bae9-52d6b8440d1d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:05.534659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed044a89-fb20-4fba-9140-dd1805ddf946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:05.538771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed044a89-fb20-4fba-9140-dd1805ddf946 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:05.538771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed044a89-fb20-4fba-9140-dd1805ddf946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:05.538771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed044a89-fb20-4fba-9140-dd1805ddf946 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:05.538771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed044a89-fb20-4fba-9140-dd1805ddf946 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:05.538771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed044a89-fb20-4fba-9140-dd1805ddf946 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:05.540751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed044a89-fb20-4fba-9140-dd1805ddf946 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:05.562537 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 964/1936] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:05.566677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccbbf2-e8aa-473f-a4eb-268e0f9c9dfa None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:05.576640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccbbf2-e8aa-473f-a4eb-268e0f9c9dfa None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:05.593215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fff85f-90a3-4d55-99a1-706763941399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:05.594039 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fff85f-90a3-4d55-99a1-706763941399 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:05.594334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fff85f-90a3-4d55-99a1-706763941399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:05.594616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fff85f-90a3-4d55-99a1-706763941399 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:05.595375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fff85f-90a3-4d55-99a1-706763941399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:05.597937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fff85f-90a3-4d55-99a1-706763941399 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:05.665484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed044a89-fb20-4fba-9140-dd1805ddf946 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:05.667672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ed044a89-fb20-4fba-9140-dd1805ddf946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:05.673556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 973/1937] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:05 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:05.678482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fff85f-90a3-4d55-99a1-706763941399 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:05.690549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed044a89-fb20-4fba-9140-dd1805ddf946 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:05.798128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:05.799238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:05.799532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:05.799811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] PATH_INFO: `/v3/projects/05c72e71d89d4f68b7699689921ea528` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:05.802171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c72e71d89d4f68b7699689921ea528)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:05.803946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fff85f-90a3-4d55-99a1-706763941399 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:05.804367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:05.809657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02468a6-0c06-4a50-856a-5eb2dc159729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:05.810606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02468a6-0c06-4a50-856a-5eb2dc159729 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:05.810792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02468a6-0c06-4a50-856a-5eb2dc159729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:05.811074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02468a6-0c06-4a50-856a-5eb2dc159729 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:05.811904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02468a6-0c06-4a50-856a-5eb2dc159729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:05.814368 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02468a6-0c06-4a50-856a-5eb2dc159729 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:05.839240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 974/1938] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:05 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:05.844388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02468a6-0c06-4a50-856a-5eb2dc159729 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:05.858552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02468a6-0c06-4a50-856a-5eb2dc159729 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:05.863395 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8646-44dd-42ff-b32b-b284c81a6254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:05.864794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8646-44dd-42ff-b32b-b284c81a6254 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:05.865078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8646-44dd-42ff-b32b-b284c81a6254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:05.865352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8646-44dd-42ff-b32b-b284c81a6254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:05.866091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb8646-44dd-42ff-b32b-b284c81a6254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:05.867792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb8646-44dd-42ff-b32b-b284c81a6254 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:05.882293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 975/1939] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:06:05.886716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8646-44dd-42ff-b32b-b284c81a6254 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:05.895009 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c72e71d89d4f68b7699689921ea528', 'request_id': 'req-03c09574-f83b-4624-886f-424ed65edc33'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:05.902043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8646-44dd-42ff-b32b-b284c81a6254 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:05.904018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c72e71d89d4f68b7699689921ea528', 'request_id': 'req-03c09574-f83b-4624-886f-424ed65edc33'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:05.913994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c72e71d89d4f68b7699689921ea528', 'request_id': 'req-03c09574-f83b-4624-886f-424ed65edc33'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:05.931159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:05.932775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] The token cache is being invalidate because project 05c72e71d89d4f68b7699689921ea528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:05.933067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:05.933855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:05.936253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 965/1940] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:05 2022] DELETE /identity/v3/projects/05c72e71d89d4f68b7699689921ea528 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:05.946647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:05.947332 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:06.007096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f158b82-a4d1-49ea-8e8d-5596e3b88edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:06.008092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f158b82-a4d1-49ea-8e8d-5596e3b88edf admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:06.008554 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f158b82-a4d1-49ea-8e8d-5596e3b88edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:06.008779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:06.008911 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f158b82-a4d1-49ea-8e8d-5596e3b88edf admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:06.009678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f158b82-a4d1-49ea-8e8d-5596e3b88edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:06.011388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f158b82-a4d1-49ea-8e8d-5596e3b88edf admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:06.024511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 976/1941] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:06.028879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f158b82-a4d1-49ea-8e8d-5596e3b88edf admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:06.030895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3f158b82-a4d1-49ea-8e8d-5596e3b88edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:06.046570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c09574-f83b-4624-886f-424ed65edc33 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:06.050761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f158b82-a4d1-49ea-8e8d-5596e3b88edf admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:06.052137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:06.053079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:06.053370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:06.053665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] PATH_INFO: `/v3/users/57f404b1ea3f417881fa0a266c1aba35` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:06.055323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57f404b1ea3f417881fa0a266c1aba35)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:06.057520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:06.067998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930848c-6da4-4b71-bea2-1a8bc198da9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:06.067998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930848c-6da4-4b71-bea2-1a8bc198da9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:06.067998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930848c-6da4-4b71-bea2-1a8bc198da9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:06.067998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930848c-6da4-4b71-bea2-1a8bc198da9e admin admin] PATH_INFO: `/v3/projects/ec53ef2af50d4fd4b183df0a86073377/users/e8c750a49d1e45e08c1d89309c30b9c2/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:06.098849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930848c-6da4-4b71-bea2-1a8bc198da9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec53ef2af50d4fd4b183df0a86073377, user_id=e8c750a49d1e45e08c1d89309c30b9c2, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:06.102354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930848c-6da4-4b71-bea2-1a8bc198da9e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:06.120591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 977/1942] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:06 2022] PUT /identity/v3/projects/ec53ef2af50d4fd4b183df0a86073377/users/e8c750a49d1e45e08c1d89309c30b9c2/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:06.131807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09654e-95f8-4d67-bbf8-cc586f250c9b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:06.132266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09654e-95f8-4d67-bbf8-cc586f250c9b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:06.132636 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09654e-95f8-4d67-bbf8-cc586f250c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:06.155350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57f404b1ea3f417881fa0a266c1aba35'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:06.158025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57f404b1ea3f417881fa0a266c1aba35'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:06.172711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57f404b1ea3f417881fa0a266c1aba35'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:06.173592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57f404b1ea3f417881fa0a266c1aba35'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:06.183791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ed09654e-95f8-4d67-bbf8-cc586f250c9b None None] MFA Rules not processed for user `e8c750a49d1e45e08c1d89309c30b9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:06.185122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ed09654e-95f8-4d67-bbf8-cc586f250c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:06.185993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57f404b1ea3f417881fa0a266c1aba35'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:06.229740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 966/1943] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:05 2022] DELETE /identity/v3/users/57f404b1ea3f417881fa0a266c1aba35 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:06.288137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 978/1944] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:06.300048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:06.301194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:06.372784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a1e8d-6315-4fdc-b19e-9d574e00d7c8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:06.377017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850a00a-c899-43f3-ba04-333271fd457d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:06.377713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850a00a-c899-43f3-ba04-333271fd457d service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:06.377995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850a00a-c899-43f3-ba04-333271fd457d service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:06.378271 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850a00a-c899-43f3-ba04-333271fd457d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:06.471410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850a00a-c899-43f3-ba04-333271fd457d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:06.474999 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850a00a-c899-43f3-ba04-333271fd457d service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:06.487530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 967/1945] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:06.720141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09654e-95f8-4d67-bbf8-cc586f250c9b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:06.722736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ed09654e-95f8-4d67-bbf8-cc586f250c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:06.819652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09654e-95f8-4d67-bbf8-cc586f250c9b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:06.874067 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:06.925324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:06.926756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:06.927054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:06.927418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] PATH_INFO: `/v3/projects/31ef94d2667d42d5b0b1eca8084ecc15` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:06.928876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ef94d2667d42d5b0b1eca8084ecc15)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:06.930873 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:06.994711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ef94d2667d42d5b0b1eca8084ecc15', 'request_id': 'req-fbae1c74-2ef9-477f-b1c0-a617b07deee1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:07.001652 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ef94d2667d42d5b0b1eca8084ecc15', 'request_id': 'req-fbae1c74-2ef9-477f-b1c0-a617b07deee1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:07.010541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ef94d2667d42d5b0b1eca8084ecc15', 'request_id': 'req-fbae1c74-2ef9-477f-b1c0-a617b07deee1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:07.035354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] The token cache is being invalidate because project 31ef94d2667d42d5b0b1eca8084ecc15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:07.036193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:07.037132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-fbae1c74-2ef9-477f-b1c0-a617b07deee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:07.038807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 979/1946] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:06 2022] DELETE /identity/v3/projects/31ef94d2667d42d5b0b1eca8084ecc15 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:07.048072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb22b0-e26a-452a-8187-f4a4f55a7619 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:07.048416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb22b0-e26a-452a-8187-f4a4f55a7619 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:07.048720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb22b0-e26a-452a-8187-f4a4f55a7619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:07.076261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b9eb22b0-e26a-452a-8187-f4a4f55a7619 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:07.077621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b9eb22b0-e26a-452a-8187-f4a4f55a7619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:07.086424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:07.160075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 968/1947] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:07.169074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90d0f9-7289-4638-8010-51cc9acdd4af None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:07.169366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90d0f9-7289-4638-8010-51cc9acdd4af None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:07.169750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90d0f9-7289-4638-8010-51cc9acdd4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:07.201453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-bf90d0f9-7289-4638-8010-51cc9acdd4af None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:07.202381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-bf90d0f9-7289-4638-8010-51cc9acdd4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:07.212308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 980/1948] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:07.217996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb22b0-e26a-452a-8187-f4a4f55a7619 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:07.235729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb22b0-e26a-452a-8187-f4a4f55a7619 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:07.241068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f9103-1100-48e7-946b-df3dac1fe15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:07.241751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f9103-1100-48e7-946b-df3dac1fe15f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:07.242039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f9103-1100-48e7-946b-df3dac1fe15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:07.242354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f9103-1100-48e7-946b-df3dac1fe15f admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:07.243102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f9103-1100-48e7-946b-df3dac1fe15f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:07.244055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f9103-1100-48e7-946b-df3dac1fe15f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:07.245868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f9103-1100-48e7-946b-df3dac1fe15f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:07.264452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 969/1949] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:07.268521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90d0f9-7289-4638-8010-51cc9acdd4af None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:07.281446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90d0f9-7289-4638-8010-51cc9acdd4af None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:07.289955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cda405-400a-4d4f-8983-edb327759269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:07.290627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cda405-400a-4d4f-8983-edb327759269 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:07.291084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cda405-400a-4d4f-8983-edb327759269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:07.291373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cda405-400a-4d4f-8983-edb327759269 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:07.292254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cda405-400a-4d4f-8983-edb327759269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:07.295064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cda405-400a-4d4f-8983-edb327759269 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:07.331033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 981/1950] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:07 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:07.335290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f9103-1100-48e7-946b-df3dac1fe15f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:07.357801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f9103-1100-48e7-946b-df3dac1fe15f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:07.462597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005df295-d8ae-4ba2-9b0a-bd7be177dcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:07.463663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005df295-d8ae-4ba2-9b0a-bd7be177dcc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:07.464328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005df295-d8ae-4ba2-9b0a-bd7be177dcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:07.464922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005df295-d8ae-4ba2-9b0a-bd7be177dcc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:07.465851 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005df295-d8ae-4ba2-9b0a-bd7be177dcc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:07.470113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005df295-d8ae-4ba2-9b0a-bd7be177dcc2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:07.526320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 970/1951] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:07 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:07.530117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cda405-400a-4d4f-8983-edb327759269 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:07.543726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cda405-400a-4d4f-8983-edb327759269 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:07.550629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969a794-2eb1-4fba-8c04-344d122af2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:07.551763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969a794-2eb1-4fba-8c04-344d122af2bc admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:07.552561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969a794-2eb1-4fba-8c04-344d122af2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:07.552956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969a794-2eb1-4fba-8c04-344d122af2bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:07.554305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969a794-2eb1-4fba-8c04-344d122af2bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:07.556375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969a794-2eb1-4fba-8c04-344d122af2bc admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:07.575129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 982/1952] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:07.575996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005df295-d8ae-4ba2-9b0a-bd7be177dcc2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:07.586557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005df295-d8ae-4ba2-9b0a-bd7be177dcc2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:07.591957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f2e83-63b4-40a8-8dab-adc50e520310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:07.592692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f2e83-63b4-40a8-8dab-adc50e520310 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:07.592977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f2e83-63b4-40a8-8dab-adc50e520310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:07.593505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f2e83-63b4-40a8-8dab-adc50e520310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:07.594260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f2e83-63b4-40a8-8dab-adc50e520310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:07.595888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f2e83-63b4-40a8-8dab-adc50e520310 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:07.609745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 971/1953] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:06:07.614401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969a794-2eb1-4fba-8c04-344d122af2bc admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:07.627355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969a794-2eb1-4fba-8c04-344d122af2bc admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:07.632816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cf99a-082b-44ad-a334-62361ef49b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:07.645850 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cf99a-082b-44ad-a334-62361ef49b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:07.646374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cf99a-082b-44ad-a334-62361ef49b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:07.646674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cf99a-082b-44ad-a334-62361ef49b71 admin admin] PATH_INFO: `/v3/projects/0ad279d66db848f69065713aa5be63b1/users/4aa8a6aa3d024f4eabb235a23ad5c327/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:07.697280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cf99a-082b-44ad-a334-62361ef49b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad279d66db848f69065713aa5be63b1, user_id=4aa8a6aa3d024f4eabb235a23ad5c327, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:07.699984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cf99a-082b-44ad-a334-62361ef49b71 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:07.733930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 983/1954] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:07 2022] PUT /identity/v3/projects/0ad279d66db848f69065713aa5be63b1/users/4aa8a6aa3d024f4eabb235a23ad5c327/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:07.742830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3fb6e-4e07-4da0-bb53-6e9887123ba8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:07.742830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3fb6e-4e07-4da0-bb53-6e9887123ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:07.742830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3fb6e-4e07-4da0-bb53-6e9887123ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:07.775390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-73c3fb6e-4e07-4da0-bb53-6e9887123ba8 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:07.776843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-73c3fb6e-4e07-4da0-bb53-6e9887123ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:07.893950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 972/1955] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:07.934173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cf99a-082b-44ad-a334-62361ef49b71 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:07.936890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ae9cf99a-082b-44ad-a334-62361ef49b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:07.950145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3fb6e-4e07-4da0-bb53-6e9887123ba8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:07.953814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-73c3fb6e-4e07-4da0-bb53-6e9887123ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:07.966087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3fb6e-4e07-4da0-bb53-6e9887123ba8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:07.970894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:07.971776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:07.972055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:07.972328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] PATH_INFO: `/v3/users/b9b04954e4d64ab6859ef8508beb855f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:07.973927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9b04954e4d64ab6859ef8508beb855f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:07.976257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:08.002515 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:08.039229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cf99a-082b-44ad-a334-62361ef49b71 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:08.040426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9b04954e4d64ab6859ef8508beb855f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:08.041218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9b04954e4d64ab6859ef8508beb855f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:08.044122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea87b0-30bc-4a10-8da7-5f0341df2020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:08.045278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea87b0-30bc-4a10-8da7-5f0341df2020 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:08.045570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea87b0-30bc-4a10-8da7-5f0341df2020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:08.045851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea87b0-30bc-4a10-8da7-5f0341df2020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:08.048991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9b04954e4d64ab6859ef8508beb855f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:08.049718 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9b04954e4d64ab6859ef8508beb855f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:08.061138 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea87b0-30bc-4a10-8da7-5f0341df2020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:08.062615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9b04954e4d64ab6859ef8508beb855f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:08.062784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea87b0-30bc-4a10-8da7-5f0341df2020 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:08.070290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:08.081552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 984/1956] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:08.093189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 973/1957] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:07 2022] DELETE /identity/v3/users/b9b04954e4d64ab6859ef8508beb855f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:08.098846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38f5a-29eb-48a1-882f-58bdaea7db66 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:08.099252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38f5a-29eb-48a1-882f-58bdaea7db66 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:08.099429 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38f5a-29eb-48a1-882f-58bdaea7db66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:08.120772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7bc38f5a-29eb-48a1-882f-58bdaea7db66 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:08.122272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7bc38f5a-29eb-48a1-882f-58bdaea7db66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:08.206147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 985/1958] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:08.217546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:08.220513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:08.299182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a89a2e-d5ce-4c0d-a40c-b558978c47f9 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:08.304328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1630513-fb07-479e-8158-d6b868a63187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:08.305075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1630513-fb07-479e-8158-d6b868a63187 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:08.305367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1630513-fb07-479e-8158-d6b868a63187 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:08.305652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1630513-fb07-479e-8158-d6b868a63187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:08.324717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1630513-fb07-479e-8158-d6b868a63187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:08.326244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1630513-fb07-479e-8158-d6b868a63187 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:08.343502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 974/1959] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:09.234920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc38f5a-29eb-48a1-882f-58bdaea7db66 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:09.238542 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7bc38f5a-29eb-48a1-882f-58bdaea7db66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:09.278568 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc38f5a-29eb-48a1-882f-58bdaea7db66 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:09.300160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:09.302022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:09.302377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:09.302689 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] PATH_INFO: `/v3/projects/c5488c571c4a4fe0a7127004a2bb7b35` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:09.304126 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5488c571c4a4fe0a7127004a2bb7b35)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:09.311270 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:09.435549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5488c571c4a4fe0a7127004a2bb7b35', 'request_id': 'req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:09.445604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5488c571c4a4fe0a7127004a2bb7b35', 'request_id': 'req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:09.460656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5488c571c4a4fe0a7127004a2bb7b35', 'request_id': 'req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:09.467995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21cc98-df2c-48e4-a340-3b7af59389ae None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:09.468502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21cc98-df2c-48e4-a340-3b7af59389ae None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:09.468686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21cc98-df2c-48e4-a340-3b7af59389ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:09.477956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] The token cache is being invalidate because project c5488c571c4a4fe0a7127004a2bb7b35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:09.481469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:09.482194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:09.484231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 986/1960] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:09 2022] DELETE /identity/v3/projects/c5488c571c4a4fe0a7127004a2bb7b35 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:09.487986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:09.490362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:09.508226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6f21cc98-df2c-48e4-a340-3b7af59389ae None None] MFA Rules not processed for user `4aa8a6aa3d024f4eabb235a23ad5c327`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:09.510056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6f21cc98-df2c-48e4-a340-3b7af59389ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:09.564132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a43d0b-0a3e-423a-bf9c-0d5b8c80f93b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:09.573899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:09.575672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:09.576087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:09.576490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] PATH_INFO: `/v3/users/554dd8bbf4454009a233e5f56ddff01c` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:09.578408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=554dd8bbf4454009a233e5f56ddff01c)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:09.581564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:09.598755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 975/1961] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:09.600433 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21cc98-df2c-48e4-a340-3b7af59389ae None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:09.602208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6f21cc98-df2c-48e4-a340-3b7af59389ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:09.626055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:09.626532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21cc98-df2c-48e4-a340-3b7af59389ae None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:09.635926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f527a57-c4c1-42a1-b51f-93b31b4ba9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:09.637452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f527a57-c4c1-42a1-b51f-93b31b4ba9ae admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:09.637746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f527a57-c4c1-42a1-b51f-93b31b4ba9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:09.638109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f527a57-c4c1-42a1-b51f-93b31b4ba9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:09.638875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f527a57-c4c1-42a1-b51f-93b31b4ba9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:09.641789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f527a57-c4c1-42a1-b51f-93b31b4ba9ae admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:09.655001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '554dd8bbf4454009a233e5f56ddff01c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:09.656083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '554dd8bbf4454009a233e5f56ddff01c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:09.666298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '554dd8bbf4454009a233e5f56ddff01c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:09.667054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:09.668150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '554dd8bbf4454009a233e5f56ddff01c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:09.678935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 976/1962] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:09 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:06:09.683219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f527a57-c4c1-42a1-b51f-93b31b4ba9ae admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:09.686214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0f527a57-c4c1-42a1-b51f-93b31b4ba9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:09.687741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '554dd8bbf4454009a233e5f56ddff01c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:09.712028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 987/1963] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:09 2022] DELETE /identity/v3/users/554dd8bbf4454009a233e5f56ddff01c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:09.778484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f527a57-c4c1-42a1-b51f-93b31b4ba9ae admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:09.848976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafba29-8ff7-424f-bfbb-53984fa58314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:09.849644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafba29-8ff7-424f-bfbb-53984fa58314 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:09.849933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafba29-8ff7-424f-bfbb-53984fa58314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:09.850211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafba29-8ff7-424f-bfbb-53984fa58314 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:09.850920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafba29-8ff7-424f-bfbb-53984fa58314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:09.853390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafba29-8ff7-424f-bfbb-53984fa58314 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:09.897645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 977/1964] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:09 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:09.901595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:09.914227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b127a4-ebce-4504-9ebf-b8933a910e2e admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:09.919946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c119368-85bd-4067-819a-f1bf16ac76bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:09.921212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c119368-85bd-4067-819a-f1bf16ac76bb admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:09.921586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c119368-85bd-4067-819a-f1bf16ac76bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:09.921983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c119368-85bd-4067-819a-f1bf16ac76bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:09.923096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c119368-85bd-4067-819a-f1bf16ac76bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:09.924954 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c119368-85bd-4067-819a-f1bf16ac76bb admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:09.935819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 988/1965] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:09.939941 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafba29-8ff7-424f-bfbb-53984fa58314 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:09.951436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafba29-8ff7-424f-bfbb-53984fa58314 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:09.958140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189cfc6-9bcc-49f6-bcb1-11b40fd917e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:09.959177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189cfc6-9bcc-49f6-bcb1-11b40fd917e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:09.959556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189cfc6-9bcc-49f6-bcb1-11b40fd917e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:09.959931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189cfc6-9bcc-49f6-bcb1-11b40fd917e0 admin admin] PATH_INFO: `/v3/projects/7486bc314f684a7faef240decb27d26d/users/8721e418082d4f03a6bf27bd2c3e1dc3/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:09.975993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189cfc6-9bcc-49f6-bcb1-11b40fd917e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7486bc314f684a7faef240decb27d26d, user_id=8721e418082d4f03a6bf27bd2c3e1dc3, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:09.978812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189cfc6-9bcc-49f6-bcb1-11b40fd917e0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:09.998764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 978/1966] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:09 2022] PUT /identity/v3/projects/7486bc314f684a7faef240decb27d26d/users/8721e418082d4f03a6bf27bd2c3e1dc3/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:10.003245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c119368-85bd-4067-819a-f1bf16ac76bb admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:10.015987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189cfc6-9bcc-49f6-bcb1-11b40fd917e0 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:10.017582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c119368-85bd-4067-819a-f1bf16ac76bb admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:10.018427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6189cfc6-9bcc-49f6-bcb1-11b40fd917e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:10.097635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec614ce-f630-495e-a085-052ff122b19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:10.098610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec614ce-f630-495e-a085-052ff122b19e admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.099133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec614ce-f630-495e-a085-052ff122b19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.099427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec614ce-f630-495e-a085-052ff122b19e admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.100186 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec614ce-f630-495e-a085-052ff122b19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:10.101919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec614ce-f630-495e-a085-052ff122b19e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:10.105459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189cfc6-9bcc-49f6-bcb1-11b40fd917e0 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:10.116990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:10.117882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.118166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.118619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] PATH_INFO: `/v3/projects/0baadba2aaa04a23b6152813d58f304a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.119340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 989/1967] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:06:10.121427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0baadba2aaa04a23b6152813d58f304a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:10.123778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec614ce-f630-495e-a085-052ff122b19e admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:10.125683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:10.134720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec614ce-f630-495e-a085-052ff122b19e admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:10.140375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c568a-de51-443e-9689-876f419c1088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:10.141312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c568a-de51-443e-9689-876f419c1088 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.141594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c568a-de51-443e-9689-876f419c1088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.141874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c568a-de51-443e-9689-876f419c1088 admin admin] PATH_INFO: `/v3/projects/7486bc314f684a7faef240decb27d26d/users/8721e418082d4f03a6bf27bd2c3e1dc3/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.143913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c568a-de51-443e-9689-876f419c1088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7486bc314f684a7faef240decb27d26d, user_id=8721e418082d4f03a6bf27bd2c3e1dc3, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:10.147289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c568a-de51-443e-9689-876f419c1088 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:10.151739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:10.176489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 990/1968] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:10 2022] PUT /identity/v3/projects/7486bc314f684a7faef240decb27d26d/users/8721e418082d4f03a6bf27bd2c3e1dc3/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:10.212139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0baadba2aaa04a23b6152813d58f304a', 'request_id': 'req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:10.220542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0baadba2aaa04a23b6152813d58f304a', 'request_id': 'req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:10.231962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0baadba2aaa04a23b6152813d58f304a', 'request_id': 'req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:10.255459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] The token cache is being invalidate because project 0baadba2aaa04a23b6152813d58f304a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:10.256499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:10.258496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9f4eeb80-f9f4-4665-9751-5d7342c66fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:10.261811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 979/1969] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:10 2022] DELETE /identity/v3/projects/0baadba2aaa04a23b6152813d58f304a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:10.266503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41494a-5408-483c-a6e7-4ed488f6d950 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.266995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41494a-5408-483c-a6e7-4ed488f6d950 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.267341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41494a-5408-483c-a6e7-4ed488f6d950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.293541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-0d41494a-5408-483c-a6e7-4ed488f6d950 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:10.295317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0d41494a-5408-483c-a6e7-4ed488f6d950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:10.377919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 991/1970] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:10.386229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbf590-2333-489f-9cf2-e89c83b8a406 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.387922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbf590-2333-489f-9cf2-e89c83b8a406 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.388656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbf590-2333-489f-9cf2-e89c83b8a406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.427993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-61bbf590-2333-489f-9cf2-e89c83b8a406 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:10.429342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-61bbf590-2333-489f-9cf2-e89c83b8a406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:10.447333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 980/1971] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:10.451560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41494a-5408-483c-a6e7-4ed488f6d950 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:10.467276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41494a-5408-483c-a6e7-4ed488f6d950 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:10.480971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d10d76-42bc-43d6-9ddd-6f72382dd4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:10.481137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d10d76-42bc-43d6-9ddd-6f72382dd4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.483208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d10d76-42bc-43d6-9ddd-6f72382dd4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.483208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d10d76-42bc-43d6-9ddd-6f72382dd4f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.483491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d10d76-42bc-43d6-9ddd-6f72382dd4f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:10.483728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d10d76-42bc-43d6-9ddd-6f72382dd4f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:10.487953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d10d76-42bc-43d6-9ddd-6f72382dd4f1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:10.526717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 992/1972] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:10.530178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbf590-2333-489f-9cf2-e89c83b8a406 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:10.546380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbf590-2333-489f-9cf2-e89c83b8a406 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:10.551648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab379ed-d8af-4174-933e-a5baf1452bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:10.552334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab379ed-d8af-4174-933e-a5baf1452bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.552673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab379ed-d8af-4174-933e-a5baf1452bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.552955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab379ed-d8af-4174-933e-a5baf1452bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.553666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab379ed-d8af-4174-933e-a5baf1452bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:10.555610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab379ed-d8af-4174-933e-a5baf1452bc6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:10.592666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 981/1973] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:10 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:10.593794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d10d76-42bc-43d6-9ddd-6f72382dd4f1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:10.602827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:10.608349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d10d76-42bc-43d6-9ddd-6f72382dd4f1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:10.686291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3c1ae-e20f-46c6-96d8-8d65818c7b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:10.687236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3c1ae-e20f-46c6-96d8-8d65818c7b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.687654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3c1ae-e20f-46c6-96d8-8d65818c7b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.688378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3c1ae-e20f-46c6-96d8-8d65818c7b32 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.688989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3c1ae-e20f-46c6-96d8-8d65818c7b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:10.692351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3c1ae-e20f-46c6-96d8-8d65818c7b32 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:10.721109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 993/1974] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:10 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:06:10.727107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab379ed-d8af-4174-933e-a5baf1452bc6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:10.742137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab379ed-d8af-4174-933e-a5baf1452bc6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:10.747492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf763e-1a45-4565-835b-542c8ee273f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:10.748326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf763e-1a45-4565-835b-542c8ee273f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.748755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf763e-1a45-4565-835b-542c8ee273f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.749136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf763e-1a45-4565-835b-542c8ee273f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.749975 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf763e-1a45-4565-835b-542c8ee273f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:10.751700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf763e-1a45-4565-835b-542c8ee273f0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:10.765637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 982/1975] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:10.769349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3c1ae-e20f-46c6-96d8-8d65818c7b32 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:10.783267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3c1ae-e20f-46c6-96d8-8d65818c7b32 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:10.794106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a41b9-112d-42e6-acec-61812a6bb154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:10.795051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a41b9-112d-42e6-acec-61812a6bb154 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.795486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a41b9-112d-42e6-acec-61812a6bb154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.795941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a41b9-112d-42e6-acec-61812a6bb154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.796866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a41b9-112d-42e6-acec-61812a6bb154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:10.798662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a41b9-112d-42e6-acec-61812a6bb154 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:10.816440 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 994/1976] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:06:10.818065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf763e-1a45-4565-835b-542c8ee273f0 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:10.830805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf763e-1a45-4565-835b-542c8ee273f0 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:10.838738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce91ba3-7cdf-4226-85f9-4f21309fed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:10.840720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce91ba3-7cdf-4226-85f9-4f21309fed6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.841169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce91ba3-7cdf-4226-85f9-4f21309fed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.841364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce91ba3-7cdf-4226-85f9-4f21309fed6a admin admin] PATH_INFO: `/v3/projects/5a3a985d89094bf991e25441494878b0/users/9529ff7fa5594d5090a0a2cdd8f4d42f/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.859775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce91ba3-7cdf-4226-85f9-4f21309fed6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3a985d89094bf991e25441494878b0, user_id=9529ff7fa5594d5090a0a2cdd8f4d42f, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:10.862740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce91ba3-7cdf-4226-85f9-4f21309fed6a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:10.888885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 983/1977] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:10 2022] PUT /identity/v3/projects/5a3a985d89094bf991e25441494878b0/users/9529ff7fa5594d5090a0a2cdd8f4d42f/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:10.890604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13a5bd-b812-4223-aa12-20ff0cafa321 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:10.890946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13a5bd-b812-4223-aa12-20ff0cafa321 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:10.891271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13a5bd-b812-4223-aa12-20ff0cafa321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:10.929707 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ec13a5bd-b812-4223-aa12-20ff0cafa321 None None] MFA Rules not processed for user `9529ff7fa5594d5090a0a2cdd8f4d42f`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:10.931545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ec13a5bd-b812-4223-aa12-20ff0cafa321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:11.017084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 995/1978] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:11.021992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce91ba3-7cdf-4226-85f9-4f21309fed6a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:11.091491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87420650-e878-482b-b4bc-6262ab04aaad None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:11.091849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87420650-e878-482b-b4bc-6262ab04aaad None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:11.092200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87420650-e878-482b-b4bc-6262ab04aaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:11.099859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce91ba3-7cdf-4226-85f9-4f21309fed6a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:11.105422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3405adf-9704-4cf3-b8e1-fe2763d9befb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:11.106092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3405adf-9704-4cf3-b8e1-fe2763d9befb admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:11.106372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3405adf-9704-4cf3-b8e1-fe2763d9befb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:11.106650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3405adf-9704-4cf3-b8e1-fe2763d9befb admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:11.107350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3405adf-9704-4cf3-b8e1-fe2763d9befb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:11.109452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3405adf-9704-4cf3-b8e1-fe2763d9befb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:11.123423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-87420650-e878-482b-b4bc-6262ab04aaad None None] MFA Rules not processed for user `8721e418082d4f03a6bf27bd2c3e1dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:11.124804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-87420650-e878-482b-b4bc-6262ab04aaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:11.139912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 984/1979] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:11 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:06:11.144219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3405adf-9704-4cf3-b8e1-fe2763d9befb admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:11.182640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:11.201058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:11.217932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 996/1980] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:11.235089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87420650-e878-482b-b4bc-6262ab04aaad None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:11.236555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-87420650-e878-482b-b4bc-6262ab04aaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:11.246058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3405adf-9704-4cf3-b8e1-fe2763d9befb admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:11.251602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1c905-b062-4341-a475-e53491404b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:11.252424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1c905-b062-4341-a475-e53491404b3a admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:11.252888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1c905-b062-4341-a475-e53491404b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:11.253116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1c905-b062-4341-a475-e53491404b3a admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:11.255612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1c905-b062-4341-a475-e53491404b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:11.257019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1c905-b062-4341-a475-e53491404b3a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:11.287885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 985/1981] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:11 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:11.292952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1c905-b062-4341-a475-e53491404b3a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:11.308928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1c905-b062-4341-a475-e53491404b3a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:11.313818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc40b40-b451-4986-b6cc-15c7c01103a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:11.314492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc40b40-b451-4986-b6cc-15c7c01103a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:11.314808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc40b40-b451-4986-b6cc-15c7c01103a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:11.315061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc40b40-b451-4986-b6cc-15c7c01103a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:11.315797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc40b40-b451-4986-b6cc-15c7c01103a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:11.318174 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc40b40-b451-4986-b6cc-15c7c01103a3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:11.324810 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87420650-e878-482b-b4bc-6262ab04aaad None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:11.329960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 986/1982] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:11.334149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc40b40-b451-4986-b6cc-15c7c01103a3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:11.345190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc40b40-b451-4986-b6cc-15c7c01103a3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:11.360830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60886b25-2561-4d0d-8259-6166f53d4100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:11.363602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60886b25-2561-4d0d-8259-6166f53d4100 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:11.363774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60886b25-2561-4d0d-8259-6166f53d4100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:11.363988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60886b25-2561-4d0d-8259-6166f53d4100 admin admin] PATH_INFO: `/v3/projects/cb6eb940d0e74578ba139dc19b3a5dea/users/0b11fba371e1436392ba7bab75de1b81/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:11.381094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60886b25-2561-4d0d-8259-6166f53d4100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6eb940d0e74578ba139dc19b3a5dea, user_id=0b11fba371e1436392ba7bab75de1b81, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:11.388906 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60886b25-2561-4d0d-8259-6166f53d4100 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:11.396676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dda4ae-2e6b-4c48-9ce5-ace6c6ff801f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:11.397431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dda4ae-2e6b-4c48-9ce5-ace6c6ff801f service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:11.397722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dda4ae-2e6b-4c48-9ce5-ace6c6ff801f service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:11.398020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dda4ae-2e6b-4c48-9ce5-ace6c6ff801f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:11.400486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f7dda4ae-2e6b-4c48-9ce5-ace6c6ff801f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:11.404445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 987/1983] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:11 2022] PUT /identity/v3/projects/cb6eb940d0e74578ba139dc19b3a5dea/users/0b11fba371e1436392ba7bab75de1b81/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:11.406477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60886b25-2561-4d0d-8259-6166f53d4100 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:11.478543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dda4ae-2e6b-4c48-9ce5-ace6c6ff801f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:11.478778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60886b25-2561-4d0d-8259-6166f53d4100 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:11.480490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dda4ae-2e6b-4c48-9ce5-ace6c6ff801f service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:11.483765 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f66b9-f331-43aa-a46b-bcebb936de6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:11.484475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f66b9-f331-43aa-a46b-bcebb936de6e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:11.484765 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f66b9-f331-43aa-a46b-bcebb936de6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:11.485048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f66b9-f331-43aa-a46b-bcebb936de6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:11.485786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f66b9-f331-43aa-a46b-bcebb936de6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:11.487395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f66b9-f331-43aa-a46b-bcebb936de6e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:11.494440 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 997/1984] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:11.499938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 988/1985] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:06:11.503977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dda4ae-2e6b-4c48-9ce5-ace6c6ff801f service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:11.516567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dda4ae-2e6b-4c48-9ce5-ace6c6ff801f service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:11.521757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f77aeb-249f-4969-97df-6e002471ecbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:11.522697 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77aeb-249f-4969-97df-6e002471ecbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:11.522988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77aeb-249f-4969-97df-6e002471ecbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:11.523259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77aeb-249f-4969-97df-6e002471ecbe admin admin] PATH_INFO: `/v3/projects/cb6eb940d0e74578ba139dc19b3a5dea/users/0b11fba371e1436392ba7bab75de1b81/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:11.525278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f77aeb-249f-4969-97df-6e002471ecbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6eb940d0e74578ba139dc19b3a5dea, user_id=0b11fba371e1436392ba7bab75de1b81, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:11.528316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f77aeb-249f-4969-97df-6e002471ecbe admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:11.540990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 998/1986] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:11 2022] PUT /identity/v3/projects/cb6eb940d0e74578ba139dc19b3a5dea/users/0b11fba371e1436392ba7bab75de1b81/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:11.551003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af1656-f5a5-4dc7-8525-f89f5bfb7e05 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:11.551448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af1656-f5a5-4dc7-8525-f89f5bfb7e05 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:11.551931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af1656-f5a5-4dc7-8525-f89f5bfb7e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:11.583963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-11af1656-f5a5-4dc7-8525-f89f5bfb7e05 None None] MFA Rules not processed for user `0b11fba371e1436392ba7bab75de1b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:11.585203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-11af1656-f5a5-4dc7-8525-f89f5bfb7e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:11.686801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:11.696539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 989/1987] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:11.871571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f77aeb-249f-4969-97df-6e002471ecbe admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:11.873315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f8f77aeb-249f-4969-97df-6e002471ecbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:11.939658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f77aeb-249f-4969-97df-6e002471ecbe admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:11.943900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53823375-d10c-4497-a245-0c6b47b643e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:11.944644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53823375-d10c-4497-a245-0c6b47b643e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:11.944932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53823375-d10c-4497-a245-0c6b47b643e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:11.945207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53823375-d10c-4497-a245-0c6b47b643e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:11.950380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:12.019755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53823375-d10c-4497-a245-0c6b47b643e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:12.021783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53823375-d10c-4497-a245-0c6b47b643e8 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:12.039491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 999/1988] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:22.033682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=24 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:22.538293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af1656-f5a5-4dc7-8525-f89f5bfb7e05 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:22.549212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af1656-f5a5-4dc7-8525-f89f5bfb7e05 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:22.554830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da179f0d-8a99-4b24-9ca2-b4d72a78c08b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:22.555494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179f0d-8a99-4b24-9ca2-b4d72a78c08b service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:22.555831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179f0d-8a99-4b24-9ca2-b4d72a78c08b service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:22.556152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179f0d-8a99-4b24-9ca2-b4d72a78c08b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:22.626626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da179f0d-8a99-4b24-9ca2-b4d72a78c08b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:22.628076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da179f0d-8a99-4b24-9ca2-b4d72a78c08b service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:22.638225 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 990/1989] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:29.192018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53823375-d10c-4497-a245-0c6b47b643e8 service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:29.202316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53823375-d10c-4497-a245-0c6b47b643e8 service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:29.205973 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d24765-861e-43a7-9dd2-9e97f117ccad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:29.206612 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d24765-861e-43a7-9dd2-9e97f117ccad service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:29.206893 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d24765-861e-43a7-9dd2-9e97f117ccad service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:29.207227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d24765-861e-43a7-9dd2-9e97f117ccad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:29.217439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d24765-861e-43a7-9dd2-9e97f117ccad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:29.221123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d24765-861e-43a7-9dd2-9e97f117ccad service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:29.231776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1000/1990] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:32.635289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=30 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:36.029144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da179f0d-8a99-4b24-9ca2-b4d72a78c08b service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:36.030720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-da179f0d-8a99-4b24-9ca2-b4d72a78c08b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:36.097129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da179f0d-8a99-4b24-9ca2-b4d72a78c08b service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:36.101703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:36.102572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:36.102854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:36.103127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] PATH_INFO: `/v3/users/87ba96a5aae6459999dadaf7f20b8cb5` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:36.104300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ba96a5aae6459999dadaf7f20b8cb5)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:36.106233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:36.152249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ba96a5aae6459999dadaf7f20b8cb5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.152843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ba96a5aae6459999dadaf7f20b8cb5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.159498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ba96a5aae6459999dadaf7f20b8cb5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.160111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ba96a5aae6459999dadaf7f20b8cb5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.168154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ba96a5aae6459999dadaf7f20b8cb5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.186339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 991/1991] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:36 2022] DELETE /identity/v3/users/87ba96a5aae6459999dadaf7f20b8cb5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:36.190997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e65ac-9d92-455f-aa2b-d77c1f61ef88 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:36.191498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e65ac-9d92-455f-aa2b-d77c1f61ef88 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:36.191592 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e65ac-9d92-455f-aa2b-d77c1f61ef88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:36.209350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-810e65ac-9d92-455f-aa2b-d77c1f61ef88 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:36.210547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-810e65ac-9d92-455f-aa2b-d77c1f61ef88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:36.280483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1001/1992] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:36.291061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:36.293017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:36.353946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1998c9-ea4a-4577-857a-b28a4b9d532d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:36.357833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944bb4d-f116-4768-b1db-4fc9aa45db01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:36.358761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944bb4d-f116-4768-b1db-4fc9aa45db01 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:36.359188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944bb4d-f116-4768-b1db-4fc9aa45db01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:36.359471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944bb4d-f116-4768-b1db-4fc9aa45db01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:36.369561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944bb4d-f116-4768-b1db-4fc9aa45db01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:36.371028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944bb4d-f116-4768-b1db-4fc9aa45db01 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:36.381781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 992/1993] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:36.644172 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e65ac-9d92-455f-aa2b-d77c1f61ef88 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:36.652132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-810e65ac-9d92-455f-aa2b-d77c1f61ef88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:36.681235 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e65ac-9d92-455f-aa2b-d77c1f61ef88 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:36.688105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:36.689136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:36.689603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:36.689885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] PATH_INFO: `/v3/projects/46a6939ef23042c6abc2a3683ed9b11b` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:36.691380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a6939ef23042c6abc2a3683ed9b11b)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:36.695601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:36.809914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a6939ef23042c6abc2a3683ed9b11b', 'request_id': 'req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.816376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a6939ef23042c6abc2a3683ed9b11b', 'request_id': 'req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.827029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a6939ef23042c6abc2a3683ed9b11b', 'request_id': 'req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.841878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] The token cache is being invalidate because project 46a6939ef23042c6abc2a3683ed9b11b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:36.842171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.842901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.844436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1002/1994] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:36 2022] DELETE /identity/v3/projects/46a6939ef23042c6abc2a3683ed9b11b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:36.848511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944bb4d-f116-4768-b1db-4fc9aa45db01 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:36.859127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944bb4d-f116-4768-b1db-4fc9aa45db01 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:36.879436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:36.924793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:36.925885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:36.926178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:36.926563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] PATH_INFO: `/v3/users/d5ebcf0e844b480b9c0038bcf4ae12a3` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:36.929126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5ebcf0e844b480b9c0038bcf4ae12a3)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:36.932557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:36.980448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5ebcf0e844b480b9c0038bcf4ae12a3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.981063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5ebcf0e844b480b9c0038bcf4ae12a3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.988334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5ebcf0e844b480b9c0038bcf4ae12a3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.989001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5ebcf0e844b480b9c0038bcf4ae12a3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:36.997945 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-357398d3-32f6-44b1-9123-e7ebac94b23e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5ebcf0e844b480b9c0038bcf4ae12a3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:37.018267 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 993/1995] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:36 2022] DELETE /identity/v3/users/d5ebcf0e844b480b9c0038bcf4ae12a3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:37.318959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:37.328745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:37.351566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf85e9-2989-4cf2-a9cb-e8ba629a9cf8 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:37.464500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:37.465452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:37.465750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:37.466030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] PATH_INFO: `/v3/projects/ff15977e143b4c3ca408d57558385175` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:37.467353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff15977e143b4c3ca408d57558385175)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:37.469512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:37.488723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:37.541671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff15977e143b4c3ca408d57558385175', 'request_id': 'req-f54d3466-ead5-4a91-85ea-d1dd483ee41c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:37.550361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff15977e143b4c3ca408d57558385175', 'request_id': 'req-f54d3466-ead5-4a91-85ea-d1dd483ee41c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:37.558033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff15977e143b4c3ca408d57558385175', 'request_id': 'req-f54d3466-ead5-4a91-85ea-d1dd483ee41c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:37.574821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] The token cache is being invalidate because project ff15977e143b4c3ca408d57558385175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:37.575164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:37.575986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f54d3466-ead5-4a91-85ea-d1dd483ee41c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:37.579051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1003/1996] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:37 2022] DELETE /identity/v3/projects/ff15977e143b4c3ca408d57558385175 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:37.588717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029dd3d-8aed-49fe-a04e-f09d96d4bfa9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:37.589253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029dd3d-8aed-49fe-a04e-f09d96d4bfa9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:37.589253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029dd3d-8aed-49fe-a04e-f09d96d4bfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:37.615953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e029dd3d-8aed-49fe-a04e-f09d96d4bfa9 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:37.617374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e029dd3d-8aed-49fe-a04e-f09d96d4bfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:37.682713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 994/1997] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:37.690493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b84508-78e0-4c39-b68d-e54314b5ae66 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:37.690788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b84508-78e0-4c39-b68d-e54314b5ae66 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:37.691096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b84508-78e0-4c39-b68d-e54314b5ae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:37.709516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b1b84508-78e0-4c39-b68d-e54314b5ae66 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:37.710773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b1b84508-78e0-4c39-b68d-e54314b5ae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:37.719098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1004/1998] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:37.724742 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029dd3d-8aed-49fe-a04e-f09d96d4bfa9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:37.736637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029dd3d-8aed-49fe-a04e-f09d96d4bfa9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:37.743831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f01598-efbc-4b48-821e-45befcc4f15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:37.744523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f01598-efbc-4b48-821e-45befcc4f15c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:37.744806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f01598-efbc-4b48-821e-45befcc4f15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:37.745077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f01598-efbc-4b48-821e-45befcc4f15c admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:37.745793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f01598-efbc-4b48-821e-45befcc4f15c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:37.746093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f01598-efbc-4b48-821e-45befcc4f15c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:37.747828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f01598-efbc-4b48-821e-45befcc4f15c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:37.762159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 995/1999] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:37.766587 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b84508-78e0-4c39-b68d-e54314b5ae66 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:37.777082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b84508-78e0-4c39-b68d-e54314b5ae66 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:37.781858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a79e5d-4ea8-42b0-94c1-68b025898b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:37.782545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a79e5d-4ea8-42b0-94c1-68b025898b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:37.782974 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a79e5d-4ea8-42b0-94c1-68b025898b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:37.783263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a79e5d-4ea8-42b0-94c1-68b025898b5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:37.783987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a79e5d-4ea8-42b0-94c1-68b025898b5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:37.786214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a79e5d-4ea8-42b0-94c1-68b025898b5c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:37.815678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1005/2000] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:37 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:06:37.820299 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f01598-efbc-4b48-821e-45befcc4f15c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:37.833200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f01598-efbc-4b48-821e-45befcc4f15c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:37.872626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:37.918712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f9197-3896-4cae-8b8e-d398252e90a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:37.919421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f9197-3896-4cae-8b8e-d398252e90a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:37.919719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f9197-3896-4cae-8b8e-d398252e90a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:37.920000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f9197-3896-4cae-8b8e-d398252e90a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:37.924452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f9197-3896-4cae-8b8e-d398252e90a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:37.924452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f9197-3896-4cae-8b8e-d398252e90a8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:37.954159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 996/2001] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:37 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:37.958744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a79e5d-4ea8-42b0-94c1-68b025898b5c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:37.977530 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a79e5d-4ea8-42b0-94c1-68b025898b5c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:37.984987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a65318-ee2a-4ae2-9cb4-e11af19c7076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:37.986300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a65318-ee2a-4ae2-9cb4-e11af19c7076 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:37.986856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a65318-ee2a-4ae2-9cb4-e11af19c7076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:37.987475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a65318-ee2a-4ae2-9cb4-e11af19c7076 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:37.988365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a65318-ee2a-4ae2-9cb4-e11af19c7076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:37.991549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a65318-ee2a-4ae2-9cb4-e11af19c7076 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:38.016315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1006/2002] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:38.020709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f9197-3896-4cae-8b8e-d398252e90a8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:38.033976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f9197-3896-4cae-8b8e-d398252e90a8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:38.043141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bef38f-ee73-41f7-8b83-d459ebd9a80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:38.043896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bef38f-ee73-41f7-8b83-d459ebd9a80f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:38.044208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bef38f-ee73-41f7-8b83-d459ebd9a80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:38.044546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bef38f-ee73-41f7-8b83-d459ebd9a80f admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:38.045401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bef38f-ee73-41f7-8b83-d459ebd9a80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:38.048344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bef38f-ee73-41f7-8b83-d459ebd9a80f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:38.065379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 997/2003] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:38.070164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a65318-ee2a-4ae2-9cb4-e11af19c7076 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:38.086199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a65318-ee2a-4ae2-9cb4-e11af19c7076 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:38.093470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1584ad-4d70-4922-8943-e6f1f441ef6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:38.094549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1584ad-4d70-4922-8943-e6f1f441ef6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:38.094894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1584ad-4d70-4922-8943-e6f1f441ef6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:38.095129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1584ad-4d70-4922-8943-e6f1f441ef6a admin admin] PATH_INFO: `/v3/projects/e1346fb6de404a26b3307945c21e9c1d/users/4db0e9d60ae348269a5a9ad7683121ed/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:38.126910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1584ad-4d70-4922-8943-e6f1f441ef6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1346fb6de404a26b3307945c21e9c1d, user_id=4db0e9d60ae348269a5a9ad7683121ed, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:38.130969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1584ad-4d70-4922-8943-e6f1f441ef6a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:38.148779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1007/2004] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:38 2022] PUT /identity/v3/projects/e1346fb6de404a26b3307945c21e9c1d/users/4db0e9d60ae348269a5a9ad7683121ed/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:38.161455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4304e8-5d2a-473c-8b37-a39d0f608b46 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:38.162653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4304e8-5d2a-473c-8b37-a39d0f608b46 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:38.163232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4304e8-5d2a-473c-8b37-a39d0f608b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:38.203529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-db4304e8-5d2a-473c-8b37-a39d0f608b46 None None] MFA Rules not processed for user `4db0e9d60ae348269a5a9ad7683121ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:38.205985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-db4304e8-5d2a-473c-8b37-a39d0f608b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:38.307012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 998/2005] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:38.312961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1584ad-4d70-4922-8943-e6f1f441ef6a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:38.380929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1584ad-4d70-4922-8943-e6f1f441ef6a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:38.387100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb08ef-723a-4dc0-a33e-092088810060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:38.387773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb08ef-723a-4dc0-a33e-092088810060 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:38.388065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb08ef-723a-4dc0-a33e-092088810060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:38.388355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb08ef-723a-4dc0-a33e-092088810060 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:38.389192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb08ef-723a-4dc0-a33e-092088810060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:38.391114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb08ef-723a-4dc0-a33e-092088810060 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:38.421110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1008/2006] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:38 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:38.426291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4304e8-5d2a-473c-8b37-a39d0f608b46 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:38.450214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4304e8-5d2a-473c-8b37-a39d0f608b46 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:38.523054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dd25b-91fd-4e8f-8f64-f66d5094557d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:38.523744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dd25b-91fd-4e8f-8f64-f66d5094557d admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:38.524032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dd25b-91fd-4e8f-8f64-f66d5094557d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:38.524311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dd25b-91fd-4e8f-8f64-f66d5094557d admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:38.525039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894dd25b-91fd-4e8f-8f64-f66d5094557d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:38.526994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894dd25b-91fd-4e8f-8f64-f66d5094557d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:38.548718 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 999/2007] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:38 2022] POST /identity/v3/users => generated 301 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:38.553069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb08ef-723a-4dc0-a33e-092088810060 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:38.563427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb08ef-723a-4dc0-a33e-092088810060 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:38.568134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18ada4-9240-489e-ae4d-6d6cefbc0060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:38.568834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18ada4-9240-489e-ae4d-6d6cefbc0060 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:38.569123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18ada4-9240-489e-ae4d-6d6cefbc0060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:38.569402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18ada4-9240-489e-ae4d-6d6cefbc0060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:38.570139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18ada4-9240-489e-ae4d-6d6cefbc0060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:38.571783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18ada4-9240-489e-ae4d-6d6cefbc0060 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:38.583293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1009/2008] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:38.588163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dd25b-91fd-4e8f-8f64-f66d5094557d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:38.598354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dd25b-91fd-4e8f-8f64-f66d5094557d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:38.614297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09037f-a4d6-4b55-8c4f-3e9db4361fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:38.615453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09037f-a4d6-4b55-8c4f-3e9db4361fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:38.615803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09037f-a4d6-4b55-8c4f-3e9db4361fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:38.616202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09037f-a4d6-4b55-8c4f-3e9db4361fc5 admin admin] PATH_INFO: `/v3/projects/5de8d152464f4c1dbb94638c14a8687b/users/0b372b01760340388f2b293511408a32/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:38.633057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09037f-a4d6-4b55-8c4f-3e9db4361fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de8d152464f4c1dbb94638c14a8687b, user_id=0b372b01760340388f2b293511408a32, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:38.636757 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09037f-a4d6-4b55-8c4f-3e9db4361fc5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:38.658761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1000/2009] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:38 2022] PUT /identity/v3/projects/5de8d152464f4c1dbb94638c14a8687b/users/0b372b01760340388f2b293511408a32/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:38.676540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18ada4-9240-489e-ae4d-6d6cefbc0060 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:38.698935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18ada4-9240-489e-ae4d-6d6cefbc0060 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:38.712553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:38.801526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8969644-6fe4-471f-850f-e62e204f1ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:38.802825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969644-6fe4-471f-850f-e62e204f1ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:38.803264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969644-6fe4-471f-850f-e62e204f1ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:38.803386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969644-6fe4-471f-850f-e62e204f1ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:38.804123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8969644-6fe4-471f-850f-e62e204f1ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:38.805909 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8969644-6fe4-471f-850f-e62e204f1ea7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:38.823715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1010/2010] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:38.825379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09037f-a4d6-4b55-8c4f-3e9db4361fc5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:38.831125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:38.837771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09037f-a4d6-4b55-8c4f-3e9db4361fc5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:38.844054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb74b5a-dde7-4e99-b1d4-19d2016c811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:38.845202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb74b5a-dde7-4e99-b1d4-19d2016c811f admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:38.845568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb74b5a-dde7-4e99-b1d4-19d2016c811f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:38.845914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb74b5a-dde7-4e99-b1d4-19d2016c811f admin admin] PATH_INFO: `/v3/projects/5de8d152464f4c1dbb94638c14a8687b/users/0b372b01760340388f2b293511408a32/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:38.848255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb74b5a-dde7-4e99-b1d4-19d2016c811f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de8d152464f4c1dbb94638c14a8687b, user_id=0b372b01760340388f2b293511408a32, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:38.852314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb74b5a-dde7-4e99-b1d4-19d2016c811f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:38.863456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1001/2011] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:38 2022] PUT /identity/v3/projects/5de8d152464f4c1dbb94638c14a8687b/users/0b372b01760340388f2b293511408a32/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:38.874065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985373f7-a9be-4cbd-b286-c120e48d191b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:38.874358 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985373f7-a9be-4cbd-b286-c120e48d191b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:38.874644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985373f7-a9be-4cbd-b286-c120e48d191b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:38.903319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-985373f7-a9be-4cbd-b286-c120e48d191b None None] MFA Rules not processed for user `0b372b01760340388f2b293511408a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:38.904674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-985373f7-a9be-4cbd-b286-c120e48d191b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:38.994556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1011/2012] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:39.006703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb74b5a-dde7-4e99-b1d4-19d2016c811f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:39.008480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1bb74b5a-dde7-4e99-b1d4-19d2016c811f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:39.069343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb74b5a-dde7-4e99-b1d4-19d2016c811f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:39.074256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17926c18-086e-4d39-8fbe-ebbdae8fcc76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:39.075376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17926c18-086e-4d39-8fbe-ebbdae8fcc76 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:39.075672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17926c18-086e-4d39-8fbe-ebbdae8fcc76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:39.075957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17926c18-086e-4d39-8fbe-ebbdae8fcc76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:39.089101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17926c18-086e-4d39-8fbe-ebbdae8fcc76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:39.090542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17926c18-086e-4d39-8fbe-ebbdae8fcc76 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:39.102607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1002/2013] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:39.232573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985373f7-a9be-4cbd-b286-c120e48d191b None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:39.298637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985373f7-a9be-4cbd-b286-c120e48d191b None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:39.303996 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:39.304936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:39.305352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:39.305636 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] PATH_INFO: `/v3/users/4db0e9d60ae348269a5a9ad7683121ed` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:39.306845 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db0e9d60ae348269a5a9ad7683121ed)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:39.308912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:39.349306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:39.359874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db0e9d60ae348269a5a9ad7683121ed'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:39.360653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db0e9d60ae348269a5a9ad7683121ed'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:39.368951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db0e9d60ae348269a5a9ad7683121ed'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:39.369616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db0e9d60ae348269a5a9ad7683121ed'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:39.382935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db0e9d60ae348269a5a9ad7683121ed'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:39.396506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1012/2014] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:39 2022] DELETE /identity/v3/users/4db0e9d60ae348269a5a9ad7683121ed => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:39.401097 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f5ea4-a50e-4766-b353-c231d23476c4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:39.401387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f5ea4-a50e-4766-b353-c231d23476c4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:39.401666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f5ea4-a50e-4766-b353-c231d23476c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:39.420255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-470f5ea4-a50e-4766-b353-c231d23476c4 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:39.421497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-470f5ea4-a50e-4766-b353-c231d23476c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:39.503171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1003/2015] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:39.513747 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:39.515428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:39.583173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a680c6-883e-4f88-9e75-d20123c7f329 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:39.594591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53380060-1cef-439d-bfc8-a432b94f2f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:39.594591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380060-1cef-439d-bfc8-a432b94f2f94 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:39.594591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380060-1cef-439d-bfc8-a432b94f2f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:39.594591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380060-1cef-439d-bfc8-a432b94f2f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:39.600192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53380060-1cef-439d-bfc8-a432b94f2f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:39.601680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53380060-1cef-439d-bfc8-a432b94f2f94 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:39.613919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1013/2016] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:39.895589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f5ea4-a50e-4766-b353-c231d23476c4 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:39.898232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-470f5ea4-a50e-4766-b353-c231d23476c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:39.926450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f5ea4-a50e-4766-b353-c231d23476c4 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:39.934878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:39.935760 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:39.936047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:39.936320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] PATH_INFO: `/v3/projects/e1346fb6de404a26b3307945c21e9c1d` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:39.937555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1346fb6de404a26b3307945c21e9c1d)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:39.939783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:39.988709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1346fb6de404a26b3307945c21e9c1d', 'request_id': 'req-6d25b726-86c1-41af-ad8d-8a52120dac95'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:39.995548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1346fb6de404a26b3307945c21e9c1d', 'request_id': 'req-6d25b726-86c1-41af-ad8d-8a52120dac95'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.003401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1346fb6de404a26b3307945c21e9c1d', 'request_id': 'req-6d25b726-86c1-41af-ad8d-8a52120dac95'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.016755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] The token cache is being invalidate because project e1346fb6de404a26b3307945c21e9c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:40.016986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.017611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.019148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1004/2017] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:39 2022] DELETE /identity/v3/projects/e1346fb6de404a26b3307945c21e9c1d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:40.023818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53380060-1cef-439d-bfc8-a432b94f2f94 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:40.034243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53380060-1cef-439d-bfc8-a432b94f2f94 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:40.098094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:40.099039 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:40.099389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:40.099666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] PATH_INFO: `/v3/users/0b372b01760340388f2b293511408a32` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:40.100867 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b372b01760340388f2b293511408a32)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:40.102772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:40.147737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b372b01760340388f2b293511408a32'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.148455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b372b01760340388f2b293511408a32'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.155269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b372b01760340388f2b293511408a32'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.155920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b372b01760340388f2b293511408a32'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.157056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:40.170334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-822f6c79-9aa8-44ef-ab23-e27f0c6216fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b372b01760340388f2b293511408a32'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.183537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1014/2018] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:40 2022] DELETE /identity/v3/users/0b372b01760340388f2b293511408a32 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:40.461008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:40.462494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:40.477699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25b726-86c1-41af-ad8d-8a52120dac95 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:40.489737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:40.540199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:40.541111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:40.541393 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:40.541668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] PATH_INFO: `/v3/projects/5de8d152464f4c1dbb94638c14a8687b` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:40.542847 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de8d152464f4c1dbb94638c14a8687b)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:40.544815 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:40.590810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de8d152464f4c1dbb94638c14a8687b', 'request_id': 'req-01b86831-e189-4a53-b942-7580e5bff656'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.596828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de8d152464f4c1dbb94638c14a8687b', 'request_id': 'req-01b86831-e189-4a53-b942-7580e5bff656'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.604366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de8d152464f4c1dbb94638c14a8687b', 'request_id': 'req-01b86831-e189-4a53-b942-7580e5bff656'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.614462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] The token cache is being invalidate because project 5de8d152464f4c1dbb94638c14a8687b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:40.614795 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.615467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-01b86831-e189-4a53-b942-7580e5bff656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:40.617031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1005/2019] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:40 2022] DELETE /identity/v3/projects/5de8d152464f4c1dbb94638c14a8687b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:40.621882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3079737-91e0-40f2-b94d-348a5d41b76a None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:40.622327 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3079737-91e0-40f2-b94d-348a5d41b76a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:40.622447 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3079737-91e0-40f2-b94d-348a5d41b76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:40.641068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-c3079737-91e0-40f2-b94d-348a5d41b76a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:40.642277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c3079737-91e0-40f2-b94d-348a5d41b76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:40.705088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1015/2020] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:40.714861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d216899-74e9-4f3d-9149-1d440b4bb60b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:40.715152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d216899-74e9-4f3d-9149-1d440b4bb60b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:40.715440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d216899-74e9-4f3d-9149-1d440b4bb60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:40.736105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9d216899-74e9-4f3d-9149-1d440b4bb60b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:40.737337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9d216899-74e9-4f3d-9149-1d440b4bb60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:40.745636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1006/2021] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:40.749959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3079737-91e0-40f2-b94d-348a5d41b76a None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:40.760485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3079737-91e0-40f2-b94d-348a5d41b76a None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:40.765027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912fb77-5a24-4ba3-8964-3521ef827b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:40.765681 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912fb77-5a24-4ba3-8964-3521ef827b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:40.765964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912fb77-5a24-4ba3-8964-3521ef827b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:40.766243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912fb77-5a24-4ba3-8964-3521ef827b3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:40.766967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912fb77-5a24-4ba3-8964-3521ef827b3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:40.767275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912fb77-5a24-4ba3-8964-3521ef827b3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:40.768905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912fb77-5a24-4ba3-8964-3521ef827b3d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:40.780881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1016/2022] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 02:06:40.784999 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d216899-74e9-4f3d-9149-1d440b4bb60b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:40.795014 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d216899-74e9-4f3d-9149-1d440b4bb60b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:40.799739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1b272-e4e6-4fad-8628-41ffaef9ac5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:40.800411 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1b272-e4e6-4fad-8628-41ffaef9ac5f admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:40.800700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1b272-e4e6-4fad-8628-41ffaef9ac5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:40.800980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1b272-e4e6-4fad-8628-41ffaef9ac5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:40.801677 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1b272-e4e6-4fad-8628-41ffaef9ac5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:40.804310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1b272-e4e6-4fad-8628-41ffaef9ac5f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:40.829559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1007/2023] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:40 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:06:40.833929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912fb77-5a24-4ba3-8964-3521ef827b3d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:40.843756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912fb77-5a24-4ba3-8964-3521ef827b3d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:40.901408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4de27-89ac-49b8-859b-acf3b2842bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:40.902085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4de27-89ac-49b8-859b-acf3b2842bed admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:40.902379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4de27-89ac-49b8-859b-acf3b2842bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:40.902665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4de27-89ac-49b8-859b-acf3b2842bed admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:40.903377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4de27-89ac-49b8-859b-acf3b2842bed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:40.905345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4de27-89ac-49b8-859b-acf3b2842bed admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:40.931467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1017/2024] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:40 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:40.933399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1b272-e4e6-4fad-8628-41ffaef9ac5f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:40.944228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1b272-e4e6-4fad-8628-41ffaef9ac5f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:40.948811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af2ad5-faae-405e-a23e-590498fade5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:40.949743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af2ad5-faae-405e-a23e-590498fade5b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:40.950031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af2ad5-faae-405e-a23e-590498fade5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:40.950300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af2ad5-faae-405e-a23e-590498fade5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:40.951019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af2ad5-faae-405e-a23e-590498fade5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:40.952625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af2ad5-faae-405e-a23e-590498fade5b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:40.963311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1008/2025] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:40.967271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4de27-89ac-49b8-859b-acf3b2842bed admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:40.977828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4de27-89ac-49b8-859b-acf3b2842bed admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:40.982474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebf12d-19a8-4e7f-b245-8ffde0ae9687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:40.983136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebf12d-19a8-4e7f-b245-8ffde0ae9687 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:40.983418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebf12d-19a8-4e7f-b245-8ffde0ae9687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:40.983697 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebf12d-19a8-4e7f-b245-8ffde0ae9687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:40.984459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebf12d-19a8-4e7f-b245-8ffde0ae9687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:40.986063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebf12d-19a8-4e7f-b245-8ffde0ae9687 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:40.996016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1018/2026] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:40.999901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af2ad5-faae-405e-a23e-590498fade5b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.011277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af2ad5-faae-405e-a23e-590498fade5b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.015907 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f35d1-c13c-400d-8b4f-fa547ab60f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.016816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f35d1-c13c-400d-8b4f-fa547ab60f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.017098 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f35d1-c13c-400d-8b4f-fa547ab60f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.017369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f35d1-c13c-400d-8b4f-fa547ab60f84 admin admin] PATH_INFO: `/v3/projects/acc068dea15c4c9480d3f0c4cc8a8fd8/users/ea972efce8c84e10bb9e415f1b20cce5/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.032949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f35d1-c13c-400d-8b4f-fa547ab60f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc068dea15c4c9480d3f0c4cc8a8fd8, user_id=ea972efce8c84e10bb9e415f1b20cce5, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.035869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f35d1-c13c-400d-8b4f-fa547ab60f84 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.050178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1009/2027] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:40 2022] PUT /identity/v3/projects/acc068dea15c4c9480d3f0c4cc8a8fd8/users/ea972efce8c84e10bb9e415f1b20cce5/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:41.060785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044c9a6-1b14-404e-86f7-929e0d4d7e53 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.061093 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044c9a6-1b14-404e-86f7-929e0d4d7e53 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.061374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044c9a6-1b14-404e-86f7-929e0d4d7e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.093065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-4044c9a6-1b14-404e-86f7-929e0d4d7e53 None None] MFA Rules not processed for user `ea972efce8c84e10bb9e415f1b20cce5`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:41.094327 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4044c9a6-1b14-404e-86f7-929e0d4d7e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:41.130239 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:41.170752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1019/2028] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:41.175095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f35d1-c13c-400d-8b4f-fa547ab60f84 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.219490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:41.237983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f35d1-c13c-400d-8b4f-fa547ab60f84 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.242784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6d913-5dba-49ef-a814-3ee8e4234758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.243442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6d913-5dba-49ef-a814-3ee8e4234758 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.243726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6d913-5dba-49ef-a814-3ee8e4234758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.244003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6d913-5dba-49ef-a814-3ee8e4234758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.244737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6d913-5dba-49ef-a814-3ee8e4234758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.246638 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6d913-5dba-49ef-a814-3ee8e4234758 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.271790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1010/2029] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:41 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:41.275860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044c9a6-1b14-404e-86f7-929e0d4d7e53 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.287243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044c9a6-1b14-404e-86f7-929e0d4d7e53 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.343358 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a68d38-4a20-4bb5-9966-85cd4a4166ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.344026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a68d38-4a20-4bb5-9966-85cd4a4166ad admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.344306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a68d38-4a20-4bb5-9966-85cd4a4166ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.344612 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a68d38-4a20-4bb5-9966-85cd4a4166ad admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.345344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a68d38-4a20-4bb5-9966-85cd4a4166ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.347258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a68d38-4a20-4bb5-9966-85cd4a4166ad admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.370961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1020/2030] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:41 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:06:41.372145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6d913-5dba-49ef-a814-3ee8e4234758 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.382623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6d913-5dba-49ef-a814-3ee8e4234758 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.387293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5bee-e3cf-4dc3-b3dc-62cc09db28b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.387949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5bee-e3cf-4dc3-b3dc-62cc09db28b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.388232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5bee-e3cf-4dc3-b3dc-62cc09db28b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.388547 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5bee-e3cf-4dc3-b3dc-62cc09db28b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.389298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d5bee-e3cf-4dc3-b3dc-62cc09db28b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.390920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d5bee-e3cf-4dc3-b3dc-62cc09db28b0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.401416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1011/2031] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:06:41.405187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a68d38-4a20-4bb5-9966-85cd4a4166ad admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.415891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a68d38-4a20-4bb5-9966-85cd4a4166ad admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.420424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501ea34-84f3-480b-bddf-883588eeb325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.421097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501ea34-84f3-480b-bddf-883588eeb325 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.421383 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501ea34-84f3-480b-bddf-883588eeb325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.421662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501ea34-84f3-480b-bddf-883588eeb325 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.422386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501ea34-84f3-480b-bddf-883588eeb325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.423971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501ea34-84f3-480b-bddf-883588eeb325 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.434507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1021/2032] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:41.438639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5bee-e3cf-4dc3-b3dc-62cc09db28b0 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.448615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5bee-e3cf-4dc3-b3dc-62cc09db28b0 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.453274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11717d-6814-4d78-a4cb-5969e799b6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.454223 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11717d-6814-4d78-a4cb-5969e799b6b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.454875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11717d-6814-4d78-a4cb-5969e799b6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.455152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11717d-6814-4d78-a4cb-5969e799b6b6 admin admin] PATH_INFO: `/v3/projects/294e62c787ae4113bc360bdd5b6a9a82/users/de6b15b5368f412f9f5ffeb3f71aa1d4/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.470013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11717d-6814-4d78-a4cb-5969e799b6b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=294e62c787ae4113bc360bdd5b6a9a82, user_id=de6b15b5368f412f9f5ffeb3f71aa1d4, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.472647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11717d-6814-4d78-a4cb-5969e799b6b6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.486314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1012/2033] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:41 2022] PUT /identity/v3/projects/294e62c787ae4113bc360bdd5b6a9a82/users/de6b15b5368f412f9f5ffeb3f71aa1d4/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:41.497344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7fd49-07f9-4624-b11a-28f764f0193a None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.497640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7fd49-07f9-4624-b11a-28f764f0193a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.497948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7fd49-07f9-4624-b11a-28f764f0193a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.525228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-36a7fd49-07f9-4624-b11a-28f764f0193a None None] MFA Rules not processed for user `de6b15b5368f412f9f5ffeb3f71aa1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:41.527057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-36a7fd49-07f9-4624-b11a-28f764f0193a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:41.592797 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1022/2034] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:41.597163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11717d-6814-4d78-a4cb-5969e799b6b6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.659349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11717d-6814-4d78-a4cb-5969e799b6b6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.665035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72aecc6-1213-4f60-a79c-d6d6e0f97389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.665690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72aecc6-1213-4f60-a79c-d6d6e0f97389 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.665974 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72aecc6-1213-4f60-a79c-d6d6e0f97389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.666253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72aecc6-1213-4f60-a79c-d6d6e0f97389 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.666939 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72aecc6-1213-4f60-a79c-d6d6e0f97389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.668897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72aecc6-1213-4f60-a79c-d6d6e0f97389 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.693724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1013/2035] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:41 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:41.697797 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7fd49-07f9-4624-b11a-28f764f0193a None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.708797 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7fd49-07f9-4624-b11a-28f764f0193a None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.747305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:41.773853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98910555-f066-4602-9a2f-e059cff7c3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.774531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98910555-f066-4602-9a2f-e059cff7c3db admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.774819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98910555-f066-4602-9a2f-e059cff7c3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.775101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98910555-f066-4602-9a2f-e059cff7c3db admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.775807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98910555-f066-4602-9a2f-e059cff7c3db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.777751 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98910555-f066-4602-9a2f-e059cff7c3db admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.801820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1023/2036] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:41 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:41.803781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72aecc6-1213-4f60-a79c-d6d6e0f97389 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.814576 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72aecc6-1213-4f60-a79c-d6d6e0f97389 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.820455 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24387f85-e02a-4549-929b-3d4c77cdc6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.821093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24387f85-e02a-4549-929b-3d4c77cdc6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.821380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24387f85-e02a-4549-929b-3d4c77cdc6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.821656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24387f85-e02a-4549-929b-3d4c77cdc6e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.822529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24387f85-e02a-4549-929b-3d4c77cdc6e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.824306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24387f85-e02a-4549-929b-3d4c77cdc6e7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.837875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1014/2037] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:06:41.842149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98910555-f066-4602-9a2f-e059cff7c3db admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.852592 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98910555-f066-4602-9a2f-e059cff7c3db admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.857136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649c584-5148-4dbf-bf7f-b2bdae65442f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.858024 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649c584-5148-4dbf-bf7f-b2bdae65442f admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.858344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649c584-5148-4dbf-bf7f-b2bdae65442f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.858611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649c584-5148-4dbf-bf7f-b2bdae65442f admin admin] PATH_INFO: `/v3/projects/ecd52cb15df64b0c90810a727876ec59/users/a79fee33707c4acdb8178e33b814fb2a/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.875289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649c584-5148-4dbf-bf7f-b2bdae65442f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd52cb15df64b0c90810a727876ec59, user_id=a79fee33707c4acdb8178e33b814fb2a, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.878035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649c584-5148-4dbf-bf7f-b2bdae65442f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.891192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1024/2038] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:41 2022] PUT /identity/v3/projects/ecd52cb15df64b0c90810a727876ec59/users/a79fee33707c4acdb8178e33b814fb2a/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:41.895221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24387f85-e02a-4549-929b-3d4c77cdc6e7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:41.905494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24387f85-e02a-4549-929b-3d4c77cdc6e7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:41.969239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996b35b-1d29-4477-a741-c1c5cf2d0c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:41.970301 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996b35b-1d29-4477-a741-c1c5cf2d0c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:41.970592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996b35b-1d29-4477-a741-c1c5cf2d0c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:41.971003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996b35b-1d29-4477-a741-c1c5cf2d0c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:41.971667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996b35b-1d29-4477-a741-c1c5cf2d0c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:41.973719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996b35b-1d29-4477-a741-c1c5cf2d0c4a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:41.985655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1015/2039] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:41.990356 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649c584-5148-4dbf-bf7f-b2bdae65442f admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:42.000893 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649c584-5148-4dbf-bf7f-b2bdae65442f admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:42.005690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6484d3-e152-410d-a11c-6b3483f6adc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:42.006661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6484d3-e152-410d-a11c-6b3483f6adc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:42.006951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6484d3-e152-410d-a11c-6b3483f6adc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:42.007231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6484d3-e152-410d-a11c-6b3483f6adc2 admin admin] PATH_INFO: `/v3/projects/ecd52cb15df64b0c90810a727876ec59/users/a79fee33707c4acdb8178e33b814fb2a/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:42.009274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6484d3-e152-410d-a11c-6b3483f6adc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd52cb15df64b0c90810a727876ec59, user_id=a79fee33707c4acdb8178e33b814fb2a, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:42.011934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6484d3-e152-410d-a11c-6b3483f6adc2 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:42.022477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1025/2040] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:41 2022] PUT /identity/v3/projects/ecd52cb15df64b0c90810a727876ec59/users/a79fee33707c4acdb8178e33b814fb2a/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:42.033384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c976482-e674-4848-8b9c-b9837b7dc8ec None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:42.033684 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c976482-e674-4848-8b9c-b9837b7dc8ec None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:42.033969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c976482-e674-4848-8b9c-b9837b7dc8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:42.040733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:42.063169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6c976482-e674-4848-8b9c-b9837b7dc8ec None None] MFA Rules not processed for user `a79fee33707c4acdb8178e33b814fb2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:42.064422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6c976482-e674-4848-8b9c-b9837b7dc8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:42.135612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1016/2041] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:42.147858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6484d3-e152-410d-a11c-6b3483f6adc2 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:42.153837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5f6484d3-e152-410d-a11c-6b3483f6adc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:42.220831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6484d3-e152-410d-a11c-6b3483f6adc2 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:42.229115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aaa77-2600-4b32-b5b1-9ac31b168005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:42.229857 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aaa77-2600-4b32-b5b1-9ac31b168005 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:42.230215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aaa77-2600-4b32-b5b1-9ac31b168005 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:42.230579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618aaa77-2600-4b32-b5b1-9ac31b168005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:42.310931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618aaa77-2600-4b32-b5b1-9ac31b168005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:42.312496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618aaa77-2600-4b32-b5b1-9ac31b168005 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:42.322519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1026/2042] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:06:43.650793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c976482-e674-4848-8b9c-b9837b7dc8ec None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:43.653052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6c976482-e674-4848-8b9c-b9837b7dc8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:43.761727 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c976482-e674-4848-8b9c-b9837b7dc8ec None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:43.762347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:43.763240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:43.763524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:43.763801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] PATH_INFO: `/v3/users/71ee50293b204bedb1ea03d77b940514` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:43.766831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=71ee50293b204bedb1ea03d77b940514)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:43.768912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:43.840868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71ee50293b204bedb1ea03d77b940514'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:43.841540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71ee50293b204bedb1ea03d77b940514'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:43.859393 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71ee50293b204bedb1ea03d77b940514'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:43.860085 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71ee50293b204bedb1ea03d77b940514'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:43.869907 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71ee50293b204bedb1ea03d77b940514'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:43.891604 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1017/2043] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:43 2022] DELETE /identity/v3/users/71ee50293b204bedb1ea03d77b940514 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:43.965976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aaa77-2600-4b32-b5b1-9ac31b168005 service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:43.967498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-618aaa77-2600-4b32-b5b1-9ac31b168005 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:43.979661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618aaa77-2600-4b32-b5b1-9ac31b168005 service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:44.033274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:44.043628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9fe7e-2bd0-4a4f-975f-5448a4f100c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:44.044811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9fe7e-2bd0-4a4f-975f-5448a4f100c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.045274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9fe7e-2bd0-4a4f-975f-5448a4f100c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.045718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9fe7e-2bd0-4a4f-975f-5448a4f100c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.047790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b7c9fe7e-2bd0-4a4f-975f-5448a4f100c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:44.096153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:44.097977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:44.121737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:44.144318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9fe7e-2bd0-4a4f-975f-5448a4f100c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:44.146444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9fe7e-2bd0-4a4f-975f-5448a4f100c2 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:44.156422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1027/2044] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:44.202385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fd70b-bc73-4390-a79d-bacc462e42ee admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:44.207005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:44.207863 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.208795 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.209093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] PATH_INFO: `/v3/projects/16b90d01b80e450b9357d02c245c9531` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.210266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b90d01b80e450b9357d02c245c9531)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:44.212168 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:44.258641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b90d01b80e450b9357d02c245c9531', 'request_id': 'req-6945b47c-f2a4-4629-9847-3e8f4597d3ed'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:44.265896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b90d01b80e450b9357d02c245c9531', 'request_id': 'req-6945b47c-f2a4-4629-9847-3e8f4597d3ed'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:44.275055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b90d01b80e450b9357d02c245c9531', 'request_id': 'req-6945b47c-f2a4-4629-9847-3e8f4597d3ed'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:44.286807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] The token cache is being invalidate because project 16b90d01b80e450b9357d02c245c9531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:44.287106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:44.287727 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6945b47c-f2a4-4629-9847-3e8f4597d3ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:44.289297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1018/2045] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:44 2022] DELETE /identity/v3/projects/16b90d01b80e450b9357d02c245c9531 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:44.294217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0ae8-3116-48d8-91a3-8c9bf87abf82 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.294507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0ae8-3116-48d8-91a3-8c9bf87abf82 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.294781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0ae8-3116-48d8-91a3-8c9bf87abf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.315588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-6d7e0ae8-3116-48d8-91a3-8c9bf87abf82 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:44.316849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6d7e0ae8-3116-48d8-91a3-8c9bf87abf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:44.385660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1028/2046] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:44.394236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a903a-a7bf-4681-86c8-53e53aba2330 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.394533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a903a-a7bf-4681-86c8-53e53aba2330 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.394812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a903a-a7bf-4681-86c8-53e53aba2330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.416036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-eb7a903a-a7bf-4681-86c8-53e53aba2330 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:44.417263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-eb7a903a-a7bf-4681-86c8-53e53aba2330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:44.426923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1019/2047] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:44.433116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0ae8-3116-48d8-91a3-8c9bf87abf82 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:44.444976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0ae8-3116-48d8-91a3-8c9bf87abf82 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:44.449539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eac03-636b-4e08-b515-986a94462264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:44.450192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eac03-636b-4e08-b515-986a94462264 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.450478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eac03-636b-4e08-b515-986a94462264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.450757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eac03-636b-4e08-b515-986a94462264 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.451485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eac03-636b-4e08-b515-986a94462264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:44.451794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eac03-636b-4e08-b515-986a94462264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:44.453433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eac03-636b-4e08-b515-986a94462264 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:44.466020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1029/2048] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 02:06:44.470302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a903a-a7bf-4681-86c8-53e53aba2330 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:44.479562 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a903a-a7bf-4681-86c8-53e53aba2330 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:44.484113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfd3dd-8fcc-453b-bd3b-c1f3ceed06ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:44.484797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfd3dd-8fcc-453b-bd3b-c1f3ceed06ba admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.485079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfd3dd-8fcc-453b-bd3b-c1f3ceed06ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.485355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfd3dd-8fcc-453b-bd3b-c1f3ceed06ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.486039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cfd3dd-8fcc-453b-bd3b-c1f3ceed06ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:44.487909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cfd3dd-8fcc-453b-bd3b-c1f3ceed06ba admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:44.512987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1020/2049] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:44 2022] POST /identity/v3/projects => generated 387 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:06:44.516998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eac03-636b-4e08-b515-986a94462264 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:44.527760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eac03-636b-4e08-b515-986a94462264 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:44.587525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda390a3-0b38-406c-912b-f0863373900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:44.588206 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda390a3-0b38-406c-912b-f0863373900b admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.588538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda390a3-0b38-406c-912b-f0863373900b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.588818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda390a3-0b38-406c-912b-f0863373900b admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.589528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda390a3-0b38-406c-912b-f0863373900b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:44.591467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda390a3-0b38-406c-912b-f0863373900b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:44.615409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1030/2050] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:44 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:06:44.616704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfd3dd-8fcc-453b-bd3b-c1f3ceed06ba admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:44.626946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfd3dd-8fcc-453b-bd3b-c1f3ceed06ba admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:44.631639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82b72b-0730-45f5-a91d-8aa1b8f03055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:44.632305 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82b72b-0730-45f5-a91d-8aa1b8f03055 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.632703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82b72b-0730-45f5-a91d-8aa1b8f03055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.632877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82b72b-0730-45f5-a91d-8aa1b8f03055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.633599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82b72b-0730-45f5-a91d-8aa1b8f03055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:44.635291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82b72b-0730-45f5-a91d-8aa1b8f03055 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:44.647419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1021/2051] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:44.651633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda390a3-0b38-406c-912b-f0863373900b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:44.662207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda390a3-0b38-406c-912b-f0863373900b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:44.667106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb61e10-dd60-4863-bcbf-3fd3c292b1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:44.667778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb61e10-dd60-4863-bcbf-3fd3c292b1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.668074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb61e10-dd60-4863-bcbf-3fd3c292b1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.668357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb61e10-dd60-4863-bcbf-3fd3c292b1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.669119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb61e10-dd60-4863-bcbf-3fd3c292b1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:44.670731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb61e10-dd60-4863-bcbf-3fd3c292b1d6 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:44.681594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1031/2052] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:44.685720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82b72b-0730-45f5-a91d-8aa1b8f03055 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:44.696436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82b72b-0730-45f5-a91d-8aa1b8f03055 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:44.701134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82edff6-170a-465e-b7a7-a01af6207e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:44.702172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82edff6-170a-465e-b7a7-a01af6207e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.702460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82edff6-170a-465e-b7a7-a01af6207e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.702733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82edff6-170a-465e-b7a7-a01af6207e40 admin admin] PATH_INFO: `/v3/projects/7daf0f9fd80c464ca4fbf7f2e19fccbd/users/a3b4081bebe74c839f5aa214275210b4/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.717884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82edff6-170a-465e-b7a7-a01af6207e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7daf0f9fd80c464ca4fbf7f2e19fccbd, user_id=a3b4081bebe74c839f5aa214275210b4, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:44.720558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82edff6-170a-465e-b7a7-a01af6207e40 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:44.735805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1022/2053] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:44 2022] PUT /identity/v3/projects/7daf0f9fd80c464ca4fbf7f2e19fccbd/users/a3b4081bebe74c839f5aa214275210b4/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:44.741029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9efca-84a3-409f-95dc-06f970deb985 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.741323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9efca-84a3-409f-95dc-06f970deb985 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.741602 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9efca-84a3-409f-95dc-06f970deb985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.748545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:44.760194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-99d9efca-84a3-409f-95dc-06f970deb985 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:44.761441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-99d9efca-84a3-409f-95dc-06f970deb985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:44.823975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1032/2054] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:44.833957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82edff6-170a-465e-b7a7-a01af6207e40 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:44.835703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c82edff6-170a-465e-b7a7-a01af6207e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:44.871244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:44.900832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82edff6-170a-465e-b7a7-a01af6207e40 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:44.904813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767c45d-24ff-48fe-8993-3b480602061e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:44.905467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c45d-24ff-48fe-8993-3b480602061e service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:44.905753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c45d-24ff-48fe-8993-3b480602061e service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:44.906034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c45d-24ff-48fe-8993-3b480602061e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:44.919435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767c45d-24ff-48fe-8993-3b480602061e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:44.920956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767c45d-24ff-48fe-8993-3b480602061e service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:44.931938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1023/2055] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:46.567309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9efca-84a3-409f-95dc-06f970deb985 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:46.571044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-99d9efca-84a3-409f-95dc-06f970deb985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:46.650982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9efca-84a3-409f-95dc-06f970deb985 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:46.655690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da995b7-5f9f-4139-ad4a-535a1aed7460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:46.656505 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da995b7-5f9f-4139-ad4a-535a1aed7460 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:46.656862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da995b7-5f9f-4139-ad4a-535a1aed7460 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:46.657252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da995b7-5f9f-4139-ad4a-535a1aed7460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:46.659422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8da995b7-5f9f-4139-ad4a-535a1aed7460 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:46.750599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da995b7-5f9f-4139-ad4a-535a1aed7460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:46.752082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da995b7-5f9f-4139-ad4a-535a1aed7460 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:46.763079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1033/2056] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:49.497190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fedf5a-c5a1-4599-8efb-6959de55e07e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:49.498049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fedf5a-c5a1-4599-8efb-6959de55e07e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:49.498481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fedf5a-c5a1-4599-8efb-6959de55e07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:49.529099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-75fedf5a-c5a1-4599-8efb-6959de55e07e None None] MFA Rules not processed for user `a3b4081bebe74c839f5aa214275210b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:49.530782 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-75fedf5a-c5a1-4599-8efb-6959de55e07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:49.613764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1024/2057] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:49.632475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da995b7-5f9f-4139-ad4a-535a1aed7460 service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:49.634209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8da995b7-5f9f-4139-ad4a-535a1aed7460 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:49.740515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:49.767744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da995b7-5f9f-4139-ad4a-535a1aed7460 service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:49.775444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23d354-9ff6-4de6-bc0b-70b1bf639c5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:49.776383 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23d354-9ff6-4de6-bc0b-70b1bf639c5a service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:49.776802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23d354-9ff6-4de6-bc0b-70b1bf639c5a service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:49.777192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23d354-9ff6-4de6-bc0b-70b1bf639c5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:49.788047 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23d354-9ff6-4de6-bc0b-70b1bf639c5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:49.790780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23d354-9ff6-4de6-bc0b-70b1bf639c5a service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:49.807591 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1034/2058] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:50.701341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fedf5a-c5a1-4599-8efb-6959de55e07e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:50.702891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-75fedf5a-c5a1-4599-8efb-6959de55e07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:50.723419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fedf5a-c5a1-4599-8efb-6959de55e07e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:50.737170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:50.744724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:50.745021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:50.746452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] PATH_INFO: `/v3/users/e8c750a49d1e45e08c1d89309c30b9c2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:50.749821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c750a49d1e45e08c1d89309c30b9c2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:50.754558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:50.858517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c750a49d1e45e08c1d89309c30b9c2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:50.859509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c750a49d1e45e08c1d89309c30b9c2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:50.885309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c750a49d1e45e08c1d89309c30b9c2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:50.886249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c750a49d1e45e08c1d89309c30b9c2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:50.898601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c750a49d1e45e08c1d89309c30b9c2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:50.934424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1025/2059] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:50 2022] DELETE /identity/v3/users/e8c750a49d1e45e08c1d89309c30b9c2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:50.938676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6436d1b-f850-48dd-975e-312f52bca505 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:50.939213 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6436d1b-f850-48dd-975e-312f52bca505 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:50.939295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6436d1b-f850-48dd-975e-312f52bca505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:50.975274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-d6436d1b-f850-48dd-975e-312f52bca505 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:50.976550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d6436d1b-f850-48dd-975e-312f52bca505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:51.127759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1035/2060] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:51.138505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:51.140067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:51.242555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:51.269837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9d442-b5a3-4823-ae6e-d3def2ba888a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:51.276822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eae69c-eab8-4354-bbd7-62d3b60ed091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:51.278619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eae69c-eab8-4354-bbd7-62d3b60ed091 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:51.279275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eae69c-eab8-4354-bbd7-62d3b60ed091 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:51.279740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eae69c-eab8-4354-bbd7-62d3b60ed091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:51.295329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eae69c-eab8-4354-bbd7-62d3b60ed091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:51.297016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eae69c-eab8-4354-bbd7-62d3b60ed091 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:51.312783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1026/2061] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:51.680689 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6436d1b-f850-48dd-975e-312f52bca505 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:51.682288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d6436d1b-f850-48dd-975e-312f52bca505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:51.705890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6436d1b-f850-48dd-975e-312f52bca505 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:51.715136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:51.716372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:51.716937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:51.717340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] PATH_INFO: `/v3/projects/ec53ef2af50d4fd4b183df0a86073377` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:51.720305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec53ef2af50d4fd4b183df0a86073377)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:51.736038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:51.858352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec53ef2af50d4fd4b183df0a86073377', 'request_id': 'req-56369627-673d-4eb6-9e54-1755c20b5fe0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:51.865912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec53ef2af50d4fd4b183df0a86073377', 'request_id': 'req-56369627-673d-4eb6-9e54-1755c20b5fe0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:51.879065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45fc43-bd27-4823-877e-6ab4045148d4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:51.880199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45fc43-bd27-4823-877e-6ab4045148d4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:51.880199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45fc43-bd27-4823-877e-6ab4045148d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:51.884109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec53ef2af50d4fd4b183df0a86073377', 'request_id': 'req-56369627-673d-4eb6-9e54-1755c20b5fe0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:51.907282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5c45fc43-bd27-4823-877e-6ab4045148d4 None None] MFA Rules not processed for user `a3b4081bebe74c839f5aa214275210b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:51.907436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] The token cache is being invalidate because project ec53ef2af50d4fd4b183df0a86073377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:51.907784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:51.913396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5c45fc43-bd27-4823-877e-6ab4045148d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:51.923923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56369627-673d-4eb6-9e54-1755c20b5fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:51.932859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1036/2062] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:51 2022] DELETE /identity/v3/projects/ec53ef2af50d4fd4b183df0a86073377 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:51.934462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788ea95-8e19-4872-9a9c-191f5a51b6a9 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:51.934651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788ea95-8e19-4872-9a9c-191f5a51b6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:51.934902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788ea95-8e19-4872-9a9c-191f5a51b6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:51.978297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-c788ea95-8e19-4872-9a9c-191f5a51b6a9 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:51.981716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c788ea95-8e19-4872-9a9c-191f5a51b6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:52.034618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1027/2063] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:52.123792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1037/2064] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:52.134563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83e30c-2a83-417e-9be5-d69ac539040c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:52.134831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83e30c-2a83-417e-9be5-d69ac539040c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:52.135115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83e30c-2a83-417e-9be5-d69ac539040c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:52.181070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0f83e30c-2a83-417e-9be5-d69ac539040c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:52.182979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0f83e30c-2a83-417e-9be5-d69ac539040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:52.202864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1028/2065] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:52.207547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788ea95-8e19-4872-9a9c-191f5a51b6a9 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:52.220867 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:52.230857 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788ea95-8e19-4872-9a9c-191f5a51b6a9 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:52.251349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68d75a-98b2-4db0-a450-12e93b0fa792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:52.251908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68d75a-98b2-4db0-a450-12e93b0fa792 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:52.252178 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68d75a-98b2-4db0-a450-12e93b0fa792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:52.252693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68d75a-98b2-4db0-a450-12e93b0fa792 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:52.253569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68d75a-98b2-4db0-a450-12e93b0fa792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:52.253880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68d75a-98b2-4db0-a450-12e93b0fa792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:52.258579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68d75a-98b2-4db0-a450-12e93b0fa792 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:52.278224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1038/2066] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:52.282396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83e30c-2a83-417e-9be5-d69ac539040c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:52.303278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83e30c-2a83-417e-9be5-d69ac539040c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:52.322888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d109d0c-5a89-441e-aace-b0483ae752ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:52.323692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d109d0c-5a89-441e-aace-b0483ae752ec admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:52.324042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d109d0c-5a89-441e-aace-b0483ae752ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:52.324388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d109d0c-5a89-441e-aace-b0483ae752ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:52.325222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d109d0c-5a89-441e-aace-b0483ae752ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:52.327313 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d109d0c-5a89-441e-aace-b0483ae752ec admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:52.388494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1029/2067] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:52 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:52.392133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68d75a-98b2-4db0-a450-12e93b0fa792 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:52.408754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68d75a-98b2-4db0-a450-12e93b0fa792 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:52.602625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6082b8-d6ed-4a5a-8382-241d71ae596a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:52.603744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6082b8-d6ed-4a5a-8382-241d71ae596a admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:52.604202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6082b8-d6ed-4a5a-8382-241d71ae596a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:52.604746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6082b8-d6ed-4a5a-8382-241d71ae596a admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:52.606157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6082b8-d6ed-4a5a-8382-241d71ae596a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:52.608339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6082b8-d6ed-4a5a-8382-241d71ae596a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:52.679545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1039/2068] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:52 2022] POST /identity/v3/users => generated 300 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:52.683791 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d109d0c-5a89-441e-aace-b0483ae752ec admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:52.722035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d109d0c-5a89-441e-aace-b0483ae752ec admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:52.742189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1569f08c-76b8-4db8-a7e8-8bca29ec9e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:52.743248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569f08c-76b8-4db8-a7e8-8bca29ec9e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:52.743801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569f08c-76b8-4db8-a7e8-8bca29ec9e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:52.744532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569f08c-76b8-4db8-a7e8-8bca29ec9e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:52.746106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1569f08c-76b8-4db8-a7e8-8bca29ec9e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:52.751665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1569f08c-76b8-4db8-a7e8-8bca29ec9e30 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:52.784067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1030/2069] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:06:52.786182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6082b8-d6ed-4a5a-8382-241d71ae596a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:52.797291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6082b8-d6ed-4a5a-8382-241d71ae596a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:52.806370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d4ce9-c9bf-4ed0-ad0d-e0c646a9137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:52.807443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d4ce9-c9bf-4ed0-ad0d-e0c646a9137f admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:52.807936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d4ce9-c9bf-4ed0-ad0d-e0c646a9137f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:52.808447 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d4ce9-c9bf-4ed0-ad0d-e0c646a9137f admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:52.809804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d4ce9-c9bf-4ed0-ad0d-e0c646a9137f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:52.811764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d4ce9-c9bf-4ed0-ad0d-e0c646a9137f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:52.843130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1040/2070] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:52.844618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1569f08c-76b8-4db8-a7e8-8bca29ec9e30 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:52.862048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1569f08c-76b8-4db8-a7e8-8bca29ec9e30 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:52.869285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b511875-070a-4c5d-9b25-3d9ead6ba909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:52.870649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b511875-070a-4c5d-9b25-3d9ead6ba909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:52.871229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b511875-070a-4c5d-9b25-3d9ead6ba909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:52.871717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b511875-070a-4c5d-9b25-3d9ead6ba909 admin admin] PATH_INFO: `/v3/projects/4318ce9be18545dfabe276ed4a935a5e/users/a30aec487b5a439a80962f9f76e93c4a/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:52.891285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b511875-070a-4c5d-9b25-3d9ead6ba909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4318ce9be18545dfabe276ed4a935a5e, user_id=a30aec487b5a439a80962f9f76e93c4a, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:52.894927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b511875-070a-4c5d-9b25-3d9ead6ba909 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:52.951171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1031/2071] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:52 2022] PUT /identity/v3/projects/4318ce9be18545dfabe276ed4a935a5e/users/a30aec487b5a439a80962f9f76e93c4a/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:52.962605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d101c-9070-44f5-a6e3-6ff65c40223a None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:52.962980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d101c-9070-44f5-a6e3-6ff65c40223a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:52.963769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d101c-9070-44f5-a6e3-6ff65c40223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:53.000152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-dd0d101c-9070-44f5-a6e3-6ff65c40223a None None] MFA Rules not processed for user `a30aec487b5a439a80962f9f76e93c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:53.001605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-dd0d101c-9070-44f5-a6e3-6ff65c40223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:53.114646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b511875-070a-4c5d-9b25-3d9ead6ba909 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:53.116968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1041/2072] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:53.117487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3b511875-070a-4c5d-9b25-3d9ead6ba909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:53.128672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d101c-9070-44f5-a6e3-6ff65c40223a None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:53.131624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-dd0d101c-9070-44f5-a6e3-6ff65c40223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:53.229410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d101c-9070-44f5-a6e3-6ff65c40223a None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:53.249608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8fcfac-117e-4852-b5b6-8aac81b8fce0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:53.250308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fcfac-117e-4852-b5b6-8aac81b8fce0 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:53.250597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fcfac-117e-4852-b5b6-8aac81b8fce0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:53.250878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fcfac-117e-4852-b5b6-8aac81b8fce0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:53.258351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b511875-070a-4c5d-9b25-3d9ead6ba909 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:53.266950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:53.268079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8fcfac-117e-4852-b5b6-8aac81b8fce0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:53.268209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:53.268409 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:53.268795 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] PATH_INFO: `/v3/users/fcf9a36ca46b444d88a195b9698290eb` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:53.269499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8fcfac-117e-4852-b5b6-8aac81b8fce0 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:53.270496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcf9a36ca46b444d88a195b9698290eb)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:53.273533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:53.302550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1042/2073] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:53.303738 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8fcfac-117e-4852-b5b6-8aac81b8fce0 service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:53.305306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4e8fcfac-117e-4852-b5b6-8aac81b8fce0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:53.307803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:53.335484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:53.356642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8fcfac-117e-4852-b5b6-8aac81b8fce0 service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:53.379428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcf9a36ca46b444d88a195b9698290eb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:53.381772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:53.382557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcf9a36ca46b444d88a195b9698290eb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:53.382678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:53.382898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:53.383187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] PATH_INFO: `/v3/users/ea972efce8c84e10bb9e415f1b20cce5` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:53.387226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea972efce8c84e10bb9e415f1b20cce5)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:53.389514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:53.406797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcf9a36ca46b444d88a195b9698290eb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:53.409641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcf9a36ca46b444d88a195b9698290eb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:53.429828 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-124da45d-b0f2-49fd-8c87-752e9492ebe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcf9a36ca46b444d88a195b9698290eb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:53.463653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1032/2074] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:53 2022] DELETE /identity/v3/users/fcf9a36ca46b444d88a195b9698290eb => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:53.468577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf396d-2a3a-4dd4-bfcb-42562ba497ab None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:53.472727 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf396d-2a3a-4dd4-bfcb-42562ba497ab None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:53.473881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf396d-2a3a-4dd4-bfcb-42562ba497ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:53.518744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e2cf396d-2a3a-4dd4-bfcb-42562ba497ab None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:53.520271 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e2cf396d-2a3a-4dd4-bfcb-42562ba497ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:53.581843 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea972efce8c84e10bb9e415f1b20cce5'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:53.583955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea972efce8c84e10bb9e415f1b20cce5'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:53.605841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea972efce8c84e10bb9e415f1b20cce5'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:53.606995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea972efce8c84e10bb9e415f1b20cce5'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:53.638910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e49ec5c8-985d-4205-a9d2-e4f98ea4c317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea972efce8c84e10bb9e415f1b20cce5'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:53.678559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1043/2075] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:53 2022] DELETE /identity/v3/users/ea972efce8c84e10bb9e415f1b20cce5 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:53.680372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704205f1-2203-46af-a18d-3d8748dce717 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:53.680696 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704205f1-2203-46af-a18d-3d8748dce717 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:53.680975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704205f1-2203-46af-a18d-3d8748dce717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:53.696344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1033/2076] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:53.714130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf396d-2a3a-4dd4-bfcb-42562ba497ab None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:53.716912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e2cf396d-2a3a-4dd4-bfcb-42562ba497ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:53.746062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-704205f1-2203-46af-a18d-3d8748dce717 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:53.747396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-704205f1-2203-46af-a18d-3d8748dce717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:53.872381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf396d-2a3a-4dd4-bfcb-42562ba497ab None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:53.888067 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1044/2077] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:53.898533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704205f1-2203-46af-a18d-3d8748dce717 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:53.902603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-704205f1-2203-46af-a18d-3d8748dce717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:53.960898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec7bcd-6c5d-46b2-aa0b-717d455ee8bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:53.961700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec7bcd-6c5d-46b2-aa0b-717d455ee8bc service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:53.962111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec7bcd-6c5d-46b2-aa0b-717d455ee8bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:53.963068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec7bcd-6c5d-46b2-aa0b-717d455ee8bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:53.983879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec7bcd-6c5d-46b2-aa0b-717d455ee8bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:53.988529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec7bcd-6c5d-46b2-aa0b-717d455ee8bc service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:54.000356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:54.007527 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1034/2078] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:06:54.059708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704205f1-2203-46af-a18d-3d8748dce717 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:54.066630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c47b4-9b02-4fcd-80c7-6c43229d68fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:54.068722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c47b4-9b02-4fcd-80c7-6c43229d68fe service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:54.069048 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c47b4-9b02-4fcd-80c7-6c43229d68fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:54.069335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c47b4-9b02-4fcd-80c7-6c43229d68fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:54.087011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c47b4-9b02-4fcd-80c7-6c43229d68fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:54.089127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c47b4-9b02-4fcd-80c7-6c43229d68fe service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:54.103267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1045/2079] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:54.607957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec7bcd-6c5d-46b2-aa0b-717d455ee8bc service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:54.610212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-07ec7bcd-6c5d-46b2-aa0b-717d455ee8bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:54.642588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec7bcd-6c5d-46b2-aa0b-717d455ee8bc service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:54.662591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:54.663614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:54.663963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:54.664948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] PATH_INFO: `/v3/projects/acc068dea15c4c9480d3f0c4cc8a8fd8` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:54.666674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc068dea15c4c9480d3f0c4cc8a8fd8)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:54.669631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:54.761467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c47b4-9b02-4fcd-80c7-6c43229d68fe service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:54.763913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-416c47b4-9b02-4fcd-80c7-6c43229d68fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:54.794129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c47b4-9b02-4fcd-80c7-6c43229d68fe service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:54.795511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc068dea15c4c9480d3f0c4cc8a8fd8', 'request_id': 'req-43595f62-9294-4232-af9d-342b7652f878'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:54.814493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc068dea15c4c9480d3f0c4cc8a8fd8', 'request_id': 'req-43595f62-9294-4232-af9d-342b7652f878'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:54.827437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc068dea15c4c9480d3f0c4cc8a8fd8', 'request_id': 'req-43595f62-9294-4232-af9d-342b7652f878'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:54.829918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:54.860725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] The token cache is being invalidate because project acc068dea15c4c9480d3f0c4cc8a8fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:54.861127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:54.861824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:54.870507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1035/2080] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:54 2022] DELETE /identity/v3/projects/acc068dea15c4c9480d3f0c4cc8a8fd8 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:54.873498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:54.873498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:54.907843 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:54.926057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:54.926221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:54.927078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] PATH_INFO: `/v3/projects/535b95f50fc4478da650335cd45e6caf` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:54.928878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=535b95f50fc4478da650335cd45e6caf)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:54.930500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:55.007541 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43595f62-9294-4232-af9d-342b7652f878 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:55.016520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '535b95f50fc4478da650335cd45e6caf', 'request_id': 'req-48e92efb-9c1e-4314-92f2-1487d1dd86e7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.032195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '535b95f50fc4478da650335cd45e6caf', 'request_id': 'req-48e92efb-9c1e-4314-92f2-1487d1dd86e7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.044147 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '535b95f50fc4478da650335cd45e6caf', 'request_id': 'req-48e92efb-9c1e-4314-92f2-1487d1dd86e7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.062243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] The token cache is being invalidate because project 535b95f50fc4478da650335cd45e6caf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:55.062855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.063592 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.065156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1046/2081] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:54 2022] DELETE /identity/v3/projects/535b95f50fc4478da650335cd45e6caf => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:55.065932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:55.069519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:55.095306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:55.096209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:55.096534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:55.096815 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] PATH_INFO: `/v3/users/4aa8a6aa3d024f4eabb235a23ad5c327` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:55.098143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa8a6aa3d024f4eabb235a23ad5c327)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:55.100609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:55.143958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:55.155088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e92efb-9c1e-4314-92f2-1487d1dd86e7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:55.158191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa8a6aa3d024f4eabb235a23ad5c327'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.158864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa8a6aa3d024f4eabb235a23ad5c327'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.160340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:55.161497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:55.161971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:55.162255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] PATH_INFO: `/v3/users/de6b15b5368f412f9f5ffeb3f71aa1d4` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:55.163629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de6b15b5368f412f9f5ffeb3f71aa1d4)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:55.165632 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:55.166383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa8a6aa3d024f4eabb235a23ad5c327'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.167128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa8a6aa3d024f4eabb235a23ad5c327'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.174461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa8a6aa3d024f4eabb235a23ad5c327'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.201026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1036/2082] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:54 2022] DELETE /identity/v3/users/4aa8a6aa3d024f4eabb235a23ad5c327 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:55.201866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:55.203188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:55.235496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de6b15b5368f412f9f5ffeb3f71aa1d4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.236101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de6b15b5368f412f9f5ffeb3f71aa1d4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.239665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:55.247274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de6b15b5368f412f9f5ffeb3f71aa1d4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.248975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de6b15b5368f412f9f5ffeb3f71aa1d4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.264607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de6b15b5368f412f9f5ffeb3f71aa1d4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.286163 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1047/2083] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:55 2022] DELETE /identity/v3/users/de6b15b5368f412f9f5ffeb3f71aa1d4 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:55.303966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfa374-7c14-4557-bfbd-37df79569a5d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:55.483786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:55.485685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:55.486042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:55.486378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] PATH_INFO: `/v3/users/976e0488f3e7473e848ab529ea8362b2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:55.487799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=976e0488f3e7473e848ab529ea8362b2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:55.491322 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:55.559110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '976e0488f3e7473e848ab529ea8362b2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.559870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '976e0488f3e7473e848ab529ea8362b2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.567722 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '976e0488f3e7473e848ab529ea8362b2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.568662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '976e0488f3e7473e848ab529ea8362b2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.574321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:55.578108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '976e0488f3e7473e848ab529ea8362b2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.612992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1037/2084] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:55 2022] DELETE /identity/v3/users/976e0488f3e7473e848ab529ea8362b2 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:55.629968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:55.632661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:55.650921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75163-6806-4636-9f3d-3a144b4a9218 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:55.712986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:55.714806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:55.765102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:55.767616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:55.767968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:55.768384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] PATH_INFO: `/v3/projects/0ad279d66db848f69065713aa5be63b1` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:55.771763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad279d66db848f69065713aa5be63b1)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:55.774987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:55.809272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477eb0f6-7ab6-4cd3-83b5-bc1840f97d92 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:55.815781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:55.816748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:55.817421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:55.818150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] PATH_INFO: `/v3/projects/294e62c787ae4113bc360bdd5b6a9a82` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:55.819623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] RBAC: Authorizing `identity:delete_project(project_id=294e62c787ae4113bc360bdd5b6a9a82)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:55.821815 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:55.834417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad279d66db848f69065713aa5be63b1', 'request_id': 'req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.841617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad279d66db848f69065713aa5be63b1', 'request_id': 'req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.848311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad279d66db848f69065713aa5be63b1', 'request_id': 'req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.861905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] The token cache is being invalidate because project 0ad279d66db848f69065713aa5be63b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:55.862198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.862833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.868029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1048/2085] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:55 2022] DELETE /identity/v3/projects/0ad279d66db848f69065713aa5be63b1 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:55.869096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:55.870879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:55.882383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '294e62c787ae4113bc360bdd5b6a9a82', 'request_id': 'req-8351c744-a174-4ae1-87b0-6b7c4002476a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.890124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '294e62c787ae4113bc360bdd5b6a9a82', 'request_id': 'req-8351c744-a174-4ae1-87b0-6b7c4002476a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.900841 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '294e62c787ae4113bc360bdd5b6a9a82', 'request_id': 'req-8351c744-a174-4ae1-87b0-6b7c4002476a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.920848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] The token cache is being invalidate because project 294e62c787ae4113bc360bdd5b6a9a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:55.921198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.922104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:55.924653 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1038/2086] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:55 2022] DELETE /identity/v3/projects/294e62c787ae4113bc360bdd5b6a9a82 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:55.928681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:55.930635 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:55.950813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:55.955695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfd4da-8de3-4d98-95a8-c07958f09cdc admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:56.019901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351c744-a174-4ae1-87b0-6b7c4002476a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:56.029563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:56.030143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:56.030469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.030758 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.031028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] PATH_INFO: `/v3/users/a79fee33707c4acdb8178e33b814fb2a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.031381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.031757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.032140 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] PATH_INFO: `/v3/users/8721e418082d4f03a6bf27bd2c3e1dc3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.032245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a79fee33707c4acdb8178e33b814fb2a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:56.033671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8721e418082d4f03a6bf27bd2c3e1dc3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:56.034206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:56.036142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:56.044535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:56.101096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8721e418082d4f03a6bf27bd2c3e1dc3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.101769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8721e418082d4f03a6bf27bd2c3e1dc3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.106158 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a79fee33707c4acdb8178e33b814fb2a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.106816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a79fee33707c4acdb8178e33b814fb2a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.108999 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8721e418082d4f03a6bf27bd2c3e1dc3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.109641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8721e418082d4f03a6bf27bd2c3e1dc3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.113643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a79fee33707c4acdb8178e33b814fb2a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.115145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a79fee33707c4acdb8178e33b814fb2a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.121960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8721e418082d4f03a6bf27bd2c3e1dc3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.122690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a79fee33707c4acdb8178e33b814fb2a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.135588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1049/2087] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:55 2022] DELETE /identity/v3/users/8721e418082d4f03a6bf27bd2c3e1dc3 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:56.137275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:56.138918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:56.142583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1039/2088] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:55 2022] DELETE /identity/v3/users/a79fee33707c4acdb8178e33b814fb2a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:56.227234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd634d9-2a73-49d5-9480-d6b197b93a0f admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:56.263990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:56.298671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:56.300315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.300731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.301149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] PATH_INFO: `/v3/projects/98cd7da92abf4da3a76b4b278c7dc597` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.302822 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=98cd7da92abf4da3a76b4b278c7dc597)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:56.305282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:56.371860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98cd7da92abf4da3a76b4b278c7dc597', 'request_id': 'req-3e25146d-f149-4895-a08a-ce601f6caaaf'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.378495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:56.381812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:56.383350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98cd7da92abf4da3a76b4b278c7dc597', 'request_id': 'req-3e25146d-f149-4895-a08a-ce601f6caaaf'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.397179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98cd7da92abf4da3a76b4b278c7dc597', 'request_id': 'req-3e25146d-f149-4895-a08a-ce601f6caaaf'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.399996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183ae99-da96-4618-a18d-7b2aa6b4e864 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:56.455727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] The token cache is being invalidate because project 98cd7da92abf4da3a76b4b278c7dc597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:56.455943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.459880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3e25146d-f149-4895-a08a-ce601f6caaaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.463934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1050/2089] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:56 2022] DELETE /identity/v3/projects/98cd7da92abf4da3a76b4b278c7dc597 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:56.476550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dca97f-5c69-4e74-befc-2c20485f1bd8 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.476918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dca97f-5c69-4e74-befc-2c20485f1bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.477219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dca97f-5c69-4e74-befc-2c20485f1bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.546748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:56.555214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.555500 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.555816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] PATH_INFO: `/v3/projects/7486bc314f684a7faef240decb27d26d` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.557268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7486bc314f684a7faef240decb27d26d)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:56.559630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:56.564451 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e5dca97f-5c69-4e74-befc-2c20485f1bd8 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:56.569218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e5dca97f-5c69-4e74-befc-2c20485f1bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:56.606768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1051/2090] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:56.607954 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dca97f-5c69-4e74-befc-2c20485f1bd8 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:56.611302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e5dca97f-5c69-4e74-befc-2c20485f1bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:56.636364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dca97f-5c69-4e74-befc-2c20485f1bd8 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:56.644632 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:56.647774 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.648824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.649428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] PATH_INFO: `/v3/projects/ecd52cb15df64b0c90810a727876ec59` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.654183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd52cb15df64b0c90810a727876ec59)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:56.656358 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:56.682512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7486bc314f684a7faef240decb27d26d', 'request_id': 'req-5de73aea-718e-4d1e-9144-fbb6b57b476e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.699060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7486bc314f684a7faef240decb27d26d', 'request_id': 'req-5de73aea-718e-4d1e-9144-fbb6b57b476e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.715964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7486bc314f684a7faef240decb27d26d', 'request_id': 'req-5de73aea-718e-4d1e-9144-fbb6b57b476e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.739292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] The token cache is being invalidate because project 7486bc314f684a7faef240decb27d26d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:56.739672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.740412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5de73aea-718e-4d1e-9144-fbb6b57b476e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.745534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1040/2091] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:56 2022] DELETE /identity/v3/projects/7486bc314f684a7faef240decb27d26d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:56.745534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff97d5-94c6-4169-9f88-de109e0a8cff None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.745534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff97d5-94c6-4169-9f88-de109e0a8cff None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.745534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff97d5-94c6-4169-9f88-de109e0a8cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.756501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd52cb15df64b0c90810a727876ec59', 'request_id': 'req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.777087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd52cb15df64b0c90810a727876ec59', 'request_id': 'req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.789670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd52cb15df64b0c90810a727876ec59', 'request_id': 'req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.789804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a7ff97d5-94c6-4169-9f88-de109e0a8cff None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:56.792233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a7ff97d5-94c6-4169-9f88-de109e0a8cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:56.805473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:56.806589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] The token cache is being invalidate because project ecd52cb15df64b0c90810a727876ec59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:56.806793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.808995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-83fd47a3-fa58-4e7e-9c8c-6e20e6c2589e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:56.811755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1052/2092] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:56 2022] DELETE /identity/v3/projects/ecd52cb15df64b0c90810a727876ec59 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:56.827095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afe3cd-89e9-418d-a47e-7d9d88eb75e2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.827385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afe3cd-89e9-418d-a47e-7d9d88eb75e2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.827669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afe3cd-89e9-418d-a47e-7d9d88eb75e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.849434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-35afe3cd-89e9-418d-a47e-7d9d88eb75e2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:56.850692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-35afe3cd-89e9-418d-a47e-7d9d88eb75e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:56.875518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1041/2093] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:56.878778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3781a05-5f90-4c70-b9f3-dc8a3968c176 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.878778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3781a05-5f90-4c70-b9f3-dc8a3968c176 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.878778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3781a05-5f90-4c70-b9f3-dc8a3968c176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.888902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:56.912643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b3781a05-5f90-4c70-b9f3-dc8a3968c176 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:56.914033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b3781a05-5f90-4c70-b9f3-dc8a3968c176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:56.925568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1042/2094] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:56.926859 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3781a05-5f90-4c70-b9f3-dc8a3968c176 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:56.938472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3781a05-5f90-4c70-b9f3-dc8a3968c176 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:56.944549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67a855-4c31-4eac-94e5-5e0d51dd9399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:56.945327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67a855-4c31-4eac-94e5-5e0d51dd9399 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.945616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67a855-4c31-4eac-94e5-5e0d51dd9399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.945892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67a855-4c31-4eac-94e5-5e0d51dd9399 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.946759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67a855-4c31-4eac-94e5-5e0d51dd9399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:56.947075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67a855-4c31-4eac-94e5-5e0d51dd9399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:56.948910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1053/2095] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:56.955397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67a855-4c31-4eac-94e5-5e0d51dd9399 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:56.955499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa31638-0d48-44a7-a347-992e4f3e4068 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.955499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa31638-0d48-44a7-a347-992e4f3e4068 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.955499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa31638-0d48-44a7-a347-992e4f3e4068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.964758 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1043/2096] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:56.966162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28252ba-1a02-4ff2-9bb1-dbbd83e07186 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:56.966467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28252ba-1a02-4ff2-9bb1-dbbd83e07186 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:56.966753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28252ba-1a02-4ff2-9bb1-dbbd83e07186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:56.974193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-9aa31638-0d48-44a7-a347-992e4f3e4068 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:56.975510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-9aa31638-0d48-44a7-a347-992e4f3e4068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:56.986087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1054/2097] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:56.987416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa31638-0d48-44a7-a347-992e4f3e4068 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:56.991064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d28252ba-1a02-4ff2-9bb1-dbbd83e07186 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:56.992861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d28252ba-1a02-4ff2-9bb1-dbbd83e07186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:57.000293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa31638-0d48-44a7-a347-992e4f3e4068 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.003201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1044/2098] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:57.004875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28252ba-1a02-4ff2-9bb1-dbbd83e07186 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.006059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43aece-cf55-46b8-94ad-e57879846ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.006726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43aece-cf55-46b8-94ad-e57879846ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.007418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43aece-cf55-46b8-94ad-e57879846ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.007709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43aece-cf55-46b8-94ad-e57879846ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.008445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43aece-cf55-46b8-94ad-e57879846ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.010541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43aece-cf55-46b8-94ad-e57879846ba3 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.023551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28252ba-1a02-4ff2-9bb1-dbbd83e07186 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.029133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca473f-faad-4da3-872f-bba5e89b4be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.030030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca473f-faad-4da3-872f-bba5e89b4be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.030359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca473f-faad-4da3-872f-bba5e89b4be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.030801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca473f-faad-4da3-872f-bba5e89b4be9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.031619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca473f-faad-4da3-872f-bba5e89b4be9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:57.031849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca473f-faad-4da3-872f-bba5e89b4be9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.033928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca473f-faad-4da3-872f-bba5e89b4be9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.040818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1055/2099] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:56 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:57.040818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43aece-cf55-46b8-94ad-e57879846ba3 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.047861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1045/2100] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:57.049116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca473f-faad-4da3-872f-bba5e89b4be9 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.062954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca473f-faad-4da3-872f-bba5e89b4be9 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.109793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43aece-cf55-46b8-94ad-e57879846ba3 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.114609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38598964-82a4-4100-b2b8-5a1ce4270516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.115274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38598964-82a4-4100-b2b8-5a1ce4270516 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.115845 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38598964-82a4-4100-b2b8-5a1ce4270516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.116122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38598964-82a4-4100-b2b8-5a1ce4270516 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.117037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38598964-82a4-4100-b2b8-5a1ce4270516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:57.117522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38598964-82a4-4100-b2b8-5a1ce4270516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.118965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38598964-82a4-4100-b2b8-5a1ce4270516 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.131422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fab95c-f42d-4a8a-92e0-05d21f848080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.132285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1056/2101] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:06:57.132456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fab95c-f42d-4a8a-92e0-05d21f848080 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.132744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fab95c-f42d-4a8a-92e0-05d21f848080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.133072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fab95c-f42d-4a8a-92e0-05d21f848080 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.133171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38598964-82a4-4100-b2b8-5a1ce4270516 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.133795 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fab95c-f42d-4a8a-92e0-05d21f848080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.135781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fab95c-f42d-4a8a-92e0-05d21f848080 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.145761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38598964-82a4-4100-b2b8-5a1ce4270516 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.151623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cb645-c3cf-46e0-8615-14fafd60c5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.152462 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cb645-c3cf-46e0-8615-14fafd60c5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.152848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cb645-c3cf-46e0-8615-14fafd60c5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.153285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cb645-c3cf-46e0-8615-14fafd60c5f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.154425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cb645-c3cf-46e0-8615-14fafd60c5f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.156502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cb645-c3cf-46e0-8615-14fafd60c5f7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.166008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1046/2102] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:57 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:57.166008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fab95c-f42d-4a8a-92e0-05d21f848080 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.178111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fab95c-f42d-4a8a-92e0-05d21f848080 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.183780 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5a58b-077d-4bcd-a677-ba79cdefb4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.184666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5a58b-077d-4bcd-a677-ba79cdefb4fe admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.185012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5a58b-077d-4bcd-a677-ba79cdefb4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.185336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5a58b-077d-4bcd-a677-ba79cdefb4fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.188811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1057/2103] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:57 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:57.189105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5a58b-077d-4bcd-a677-ba79cdefb4fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.189211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cb645-c3cf-46e0-8615-14fafd60c5f7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.189288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5a58b-077d-4bcd-a677-ba79cdefb4fe admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.220117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1047/2104] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:57 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:57.220117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5a58b-077d-4bcd-a677-ba79cdefb4fe admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.259295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cb645-c3cf-46e0-8615-14fafd60c5f7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.267466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:57.287213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5a58b-077d-4bcd-a677-ba79cdefb4fe admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.287574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:57.292441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f70ae-bc85-4fee-a221-52ec948c0f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.293108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f70ae-bc85-4fee-a221-52ec948c0f70 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.293490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f70ae-bc85-4fee-a221-52ec948c0f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.293823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f70ae-bc85-4fee-a221-52ec948c0f70 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.294572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f70ae-bc85-4fee-a221-52ec948c0f70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.296588 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f70ae-bc85-4fee-a221-52ec948c0f70 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.316903 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1048/2105] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:57 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:57.318039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f70ae-bc85-4fee-a221-52ec948c0f70 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.321712 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd45a8-3898-4ffc-836b-f29ae28787be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.322390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd45a8-3898-4ffc-836b-f29ae28787be admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.322679 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd45a8-3898-4ffc-836b-f29ae28787be admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.322959 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd45a8-3898-4ffc-836b-f29ae28787be admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.323692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd45a8-3898-4ffc-836b-f29ae28787be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.325337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd45a8-3898-4ffc-836b-f29ae28787be admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.328152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f70ae-bc85-4fee-a221-52ec948c0f70 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.333815 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa11278-928d-4c54-bc64-5497b0d232b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.334567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa11278-928d-4c54-bc64-5497b0d232b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.334971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa11278-928d-4c54-bc64-5497b0d232b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.335343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa11278-928d-4c54-bc64-5497b0d232b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.338925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1058/2106] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:57.339079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa11278-928d-4c54-bc64-5497b0d232b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.339193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd45a8-3898-4ffc-836b-f29ae28787be admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.339290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa11278-928d-4c54-bc64-5497b0d232b1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.346068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd45a8-3898-4ffc-836b-f29ae28787be admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.351528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483e28-e163-47ff-bc21-c7cf8051eb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.352634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483e28-e163-47ff-bc21-c7cf8051eb4f admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.352942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483e28-e163-47ff-bc21-c7cf8051eb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.353261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483e28-e163-47ff-bc21-c7cf8051eb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.353971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6483e28-e163-47ff-bc21-c7cf8051eb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.355584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6483e28-e163-47ff-bc21-c7cf8051eb4f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.360953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1049/2107] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:57 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:57.361667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa11278-928d-4c54-bc64-5497b0d232b1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.371209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1059/2108] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:57.371209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483e28-e163-47ff-bc21-c7cf8051eb4f admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.374007 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa11278-928d-4c54-bc64-5497b0d232b1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.378956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8662a9-efa2-492b-95d9-4b0137d8df5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.379138 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483e28-e163-47ff-bc21-c7cf8051eb4f admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.379680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8662a9-efa2-492b-95d9-4b0137d8df5a admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.379966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8662a9-efa2-492b-95d9-4b0137d8df5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.380242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8662a9-efa2-492b-95d9-4b0137d8df5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.381008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8662a9-efa2-492b-95d9-4b0137d8df5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.382618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8662a9-efa2-492b-95d9-4b0137d8df5a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.384300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a80b3-60b2-443d-ab62-1981688f7086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.384999 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a80b3-60b2-443d-ab62-1981688f7086 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.385285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a80b3-60b2-443d-ab62-1981688f7086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.385563 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a80b3-60b2-443d-ab62-1981688f7086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.386292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a80b3-60b2-443d-ab62-1981688f7086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.387899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a80b3-60b2-443d-ab62-1981688f7086 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.398933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1050/2109] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:57.399775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1060/2110] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:57.400707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8662a9-efa2-492b-95d9-4b0137d8df5a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.403351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a80b3-60b2-443d-ab62-1981688f7086 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.412770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8662a9-efa2-492b-95d9-4b0137d8df5a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.414277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a80b3-60b2-443d-ab62-1981688f7086 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.417495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f9dcf-7393-457a-b2ea-0fceac1eb098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.418185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9dcf-7393-457a-b2ea-0fceac1eb098 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.418655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9dcf-7393-457a-b2ea-0fceac1eb098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.418943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9dcf-7393-457a-b2ea-0fceac1eb098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.419046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6b011-3155-4773-a098-5acac535b9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.419629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6b011-3155-4773-a098-5acac535b9aa admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.419749 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f9dcf-7393-457a-b2ea-0fceac1eb098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.419919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6b011-3155-4773-a098-5acac535b9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.420304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6b011-3155-4773-a098-5acac535b9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.421468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6b011-3155-4773-a098-5acac535b9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.421603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f9dcf-7393-457a-b2ea-0fceac1eb098 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.422698 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6b011-3155-4773-a098-5acac535b9aa admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.439575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1051/2111] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:57.439575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f9dcf-7393-457a-b2ea-0fceac1eb098 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.439835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1061/2112] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:57.439835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6b011-3155-4773-a098-5acac535b9aa admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.448902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6b011-3155-4773-a098-5acac535b9aa admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.449624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f9dcf-7393-457a-b2ea-0fceac1eb098 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.453678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946d969-36d0-4163-bb81-2479aff359ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.454192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc2075-2f47-49c1-8f98-3b9b5c65668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.454556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946d969-36d0-4163-bb81-2479aff359ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.454880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946d969-36d0-4163-bb81-2479aff359ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.455065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc2075-2f47-49c1-8f98-3b9b5c65668d admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.455158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946d969-36d0-4163-bb81-2479aff359ac admin admin] PATH_INFO: `/v3/projects/540c921edbf24dbeaf13675b583bb464/users/cc19addacb9c412cb008e243c1fc7ace/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.455352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc2075-2f47-49c1-8f98-3b9b5c65668d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.455632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc2075-2f47-49c1-8f98-3b9b5c65668d admin admin] PATH_INFO: `/v3/projects/4cfe116cc6654d12b70df88a3489bd5e/users/79e64aa72b564a87bcf5013a6693dc61/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.470322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946d969-36d0-4163-bb81-2479aff359ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=540c921edbf24dbeaf13675b583bb464, user_id=cc19addacb9c412cb008e243c1fc7ace, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.470858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc2075-2f47-49c1-8f98-3b9b5c65668d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cfe116cc6654d12b70df88a3489bd5e, user_id=79e64aa72b564a87bcf5013a6693dc61, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.473199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946d969-36d0-4163-bb81-2479aff359ac admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.474431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc2075-2f47-49c1-8f98-3b9b5c65668d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.487371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1062/2113] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:57 2022] PUT /identity/v3/projects/540c921edbf24dbeaf13675b583bb464/users/cc19addacb9c412cb008e243c1fc7ace/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:57.488284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946d969-36d0-4163-bb81-2479aff359ac admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.488412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1052/2114] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:57 2022] PUT /identity/v3/projects/4cfe116cc6654d12b70df88a3489bd5e/users/79e64aa72b564a87bcf5013a6693dc61/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:57.500941 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78527ef-ad98-4bfe-be4c-1106e32020d3 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.501308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78527ef-ad98-4bfe-be4c-1106e32020d3 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.501651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78527ef-ad98-4bfe-be4c-1106e32020d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.518833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b78527ef-ad98-4bfe-be4c-1106e32020d3 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:57.520763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b78527ef-ad98-4bfe-be4c-1106e32020d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:57.550779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946d969-36d0-4163-bb81-2479aff359ac admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.581169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1053/2115] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:57.585566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de142c7e-ce48-43c6-b677-282b7636a6bd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.585865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de142c7e-ce48-43c6-b677-282b7636a6bd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.586149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de142c7e-ce48-43c6-b677-282b7636a6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.608786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0357b3c-3722-4675-8c86-86e83afad338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.609691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0357b3c-3722-4675-8c86-86e83afad338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.609975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0357b3c-3722-4675-8c86-86e83afad338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.610254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0357b3c-3722-4675-8c86-86e83afad338 admin admin] PATH_INFO: `/v3/projects/ec672910df094df5927552f16155841b/users/08e750a6bf0149d187712a5effaa6bf4/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.612553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-de142c7e-ce48-43c6-b677-282b7636a6bd None None] MFA Rules not processed for user `79e64aa72b564a87bcf5013a6693dc61`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:57.614313 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-de142c7e-ce48-43c6-b677-282b7636a6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:57.625738 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0357b3c-3722-4675-8c86-86e83afad338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec672910df094df5927552f16155841b, user_id=08e750a6bf0149d187712a5effaa6bf4, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.628572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0357b3c-3722-4675-8c86-86e83afad338 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.637631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:57.644717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1063/2116] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:57 2022] PUT /identity/v3/projects/ec672910df094df5927552f16155841b/users/08e750a6bf0149d187712a5effaa6bf4/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:57.650857 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0357b3c-3722-4675-8c86-86e83afad338 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.652945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e0357b3c-3722-4675-8c86-86e83afad338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:57.688378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:57.692900 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1054/2117] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:57.695241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693264c-6211-47ef-b49a-45fcbb795010 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.695241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693264c-6211-47ef-b49a-45fcbb795010 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.695453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693264c-6211-47ef-b49a-45fcbb795010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.712524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0357b3c-3722-4675-8c86-86e83afad338 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.716226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaedbf5-9403-4589-822c-3afbf59a0d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.717820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaedbf5-9403-4589-822c-3afbf59a0d80 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.717820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaedbf5-9403-4589-822c-3afbf59a0d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.717820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaedbf5-9403-4589-822c-3afbf59a0d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.725897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7693264c-6211-47ef-b49a-45fcbb795010 None None] MFA Rules not processed for user `08e750a6bf0149d187712a5effaa6bf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:57.727357 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7693264c-6211-47ef-b49a-45fcbb795010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:57.790621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaedbf5-9403-4589-822c-3afbf59a0d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.792258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaedbf5-9403-4589-822c-3afbf59a0d80 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.797412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1055/2118] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:57.798387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693264c-6211-47ef-b49a-45fcbb795010 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.807414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1064/2119] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:57.810200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaedbf5-9403-4589-822c-3afbf59a0d80 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.811663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-eaaedbf5-9403-4589-822c-3afbf59a0d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:57.813593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693264c-6211-47ef-b49a-45fcbb795010 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.820870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00626896-3309-408f-8352-f230ff017083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.821918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00626896-3309-408f-8352-f230ff017083 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.822297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00626896-3309-408f-8352-f230ff017083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.822833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00626896-3309-408f-8352-f230ff017083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.823535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00626896-3309-408f-8352-f230ff017083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.826108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00626896-3309-408f-8352-f230ff017083 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.861681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1056/2120] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:57 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:57.868376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00626896-3309-408f-8352-f230ff017083 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.896432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaedbf5-9403-4589-822c-3afbf59a0d80 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.947607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00626896-3309-408f-8352-f230ff017083 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:57.953156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d1c98-e25b-450e-ab06-9785f4246a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.954281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d1c98-e25b-450e-ab06-9785f4246a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.954281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d1c98-e25b-450e-ab06-9785f4246a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.954708 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d1c98-e25b-450e-ab06-9785f4246a80 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:57.955416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d1c98-e25b-450e-ab06-9785f4246a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:57.957599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d1c98-e25b-450e-ab06-9785f4246a80 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:57.983106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1057/2121] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:57 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:06:57.987048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d1c98-e25b-450e-ab06-9785f4246a80 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:57.987960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f7060-30df-433d-993a-df185814424e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:57.989471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f7060-30df-433d-993a-df185814424e service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:57.989949 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f7060-30df-433d-993a-df185814424e service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:57.992902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f7060-30df-433d-993a-df185814424e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.000638 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d1c98-e25b-450e-ab06-9785f4246a80 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:58.007464 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060192c-f632-4a29-8f6b-9acd4d0dddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:58.008514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060192c-f632-4a29-8f6b-9acd4d0dddfb admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.008711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060192c-f632-4a29-8f6b-9acd4d0dddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.008994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060192c-f632-4a29-8f6b-9acd4d0dddfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.009877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060192c-f632-4a29-8f6b-9acd4d0dddfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:58.021037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060192c-f632-4a29-8f6b-9acd4d0dddfb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:58.024722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:58.032463 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1058/2122] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:58.037381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060192c-f632-4a29-8f6b-9acd4d0dddfb admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:58.053155 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060192c-f632-4a29-8f6b-9acd4d0dddfb admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:58.058176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615143a8-1b23-4533-bfc1-baecb9ae6a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:58.059072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615143a8-1b23-4533-bfc1-baecb9ae6a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.059364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615143a8-1b23-4533-bfc1-baecb9ae6a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.059718 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615143a8-1b23-4533-bfc1-baecb9ae6a06 admin admin] PATH_INFO: `/v3/projects/bde984819d4a40e38d9554878fac56a9/users/12c6a3c5b64c49ffac9893fde0855184/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.077277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615143a8-1b23-4533-bfc1-baecb9ae6a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde984819d4a40e38d9554878fac56a9, user_id=12c6a3c5b64c49ffac9893fde0855184, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:58.079127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f7060-30df-433d-993a-df185814424e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:58.079995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615143a8-1b23-4533-bfc1-baecb9ae6a06 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:58.080608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f7060-30df-433d-993a-df185814424e service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:58.092170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1065/2123] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:06:58.097154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1059/2124] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:58 2022] PUT /identity/v3/projects/bde984819d4a40e38d9554878fac56a9/users/12c6a3c5b64c49ffac9893fde0855184/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:58.101134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f7060-30df-433d-993a-df185814424e service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:58.112705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615143a8-1b23-4533-bfc1-baecb9ae6a06 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:58.113333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f7060-30df-433d-993a-df185814424e service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:58.144345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:58.194691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615143a8-1b23-4533-bfc1-baecb9ae6a06 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:58.194882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b84e21-4cb9-41e4-9722-2cf5346d0087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:58.195633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84e21-4cb9-41e4-9722-2cf5346d0087 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.195997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84e21-4cb9-41e4-9722-2cf5346d0087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.196347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84e21-4cb9-41e4-9722-2cf5346d0087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.197325 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b84e21-4cb9-41e4-9722-2cf5346d0087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:58.199100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b84e21-4cb9-41e4-9722-2cf5346d0087 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:58.205563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:58.206461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.206744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.207025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] PATH_INFO: `/v3/users/08e750a6bf0149d187712a5effaa6bf4` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.208827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e750a6bf0149d187712a5effaa6bf4)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:58.210755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:58.214463 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1066/2125] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:06:58.218182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b84e21-4cb9-41e4-9722-2cf5346d0087 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:58.232527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b84e21-4cb9-41e4-9722-2cf5346d0087 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:58.240915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafa375-c509-4eec-93ee-b17844477816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:58.242121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa375-c509-4eec-93ee-b17844477816 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.242474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa375-c509-4eec-93ee-b17844477816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.242953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa375-c509-4eec-93ee-b17844477816 admin admin] PATH_INFO: `/v3/projects/bde984819d4a40e38d9554878fac56a9/users/12c6a3c5b64c49ffac9893fde0855184/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.245488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafa375-c509-4eec-93ee-b17844477816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde984819d4a40e38d9554878fac56a9, user_id=12c6a3c5b64c49ffac9893fde0855184, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:58.250495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafa375-c509-4eec-93ee-b17844477816 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:58.264661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1067/2126] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:58 2022] PUT /identity/v3/projects/bde984819d4a40e38d9554878fac56a9/users/12c6a3c5b64c49ffac9893fde0855184/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:58.275915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc8d17-254a-4c66-a34f-1613b595ef9f None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.276736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc8d17-254a-4c66-a34f-1613b595ef9f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.276886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc8d17-254a-4c66-a34f-1613b595ef9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.291138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e750a6bf0149d187712a5effaa6bf4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:58.291861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e750a6bf0149d187712a5effaa6bf4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:58.299386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e750a6bf0149d187712a5effaa6bf4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:58.300133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e750a6bf0149d187712a5effaa6bf4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:58.311643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67c31223-a85b-4ed7-aff4-7838f644fb67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e750a6bf0149d187712a5effaa6bf4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:58.315304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-8ffc8d17-254a-4c66-a34f-1613b595ef9f None None] MFA Rules not processed for user `12c6a3c5b64c49ffac9893fde0855184`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:58.316586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8ffc8d17-254a-4c66-a34f-1613b595ef9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:58.336805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1060/2127] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:58 2022] DELETE /identity/v3/users/08e750a6bf0149d187712a5effaa6bf4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:58.341041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f179ff0-d7ce-4495-ad6e-51752c01c71b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.341334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f179ff0-d7ce-4495-ad6e-51752c01c71b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.341617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f179ff0-d7ce-4495-ad6e-51752c01c71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.373763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8f179ff0-d7ce-4495-ad6e-51752c01c71b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:58.375178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8f179ff0-d7ce-4495-ad6e-51752c01c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:58.412009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1068/2128] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:58.423853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc8d17-254a-4c66-a34f-1613b595ef9f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:58.425365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8ffc8d17-254a-4c66-a34f-1613b595ef9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:58.449921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1061/2129] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:58.461027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:58.461224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f179ff0-d7ce-4495-ad6e-51752c01c71b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:58.463496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8f179ff0-d7ce-4495-ad6e-51752c01c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:58.494640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc8d17-254a-4c66-a34f-1613b595ef9f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:58.519688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:58.536191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f179ff0-d7ce-4495-ad6e-51752c01c71b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:58.541492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab9482-b198-4791-a718-0a7f503e3a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:58.542167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab9482-b198-4791-a718-0a7f503e3a0a service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.542469 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab9482-b198-4791-a718-0a7f503e3a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.542747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab9482-b198-4791-a718-0a7f503e3a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.553274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab9482-b198-4791-a718-0a7f503e3a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:58.553471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0396f4-7941-4707-8a4e-4483165111b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:58.554173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0396f4-7941-4707-8a4e-4483165111b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.554559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0396f4-7941-4707-8a4e-4483165111b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.554759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab9482-b198-4791-a718-0a7f503e3a0a service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:58.555169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0396f4-7941-4707-8a4e-4483165111b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.565868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1062/2130] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:06:58.630628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0396f4-7941-4707-8a4e-4483165111b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:58.632339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0396f4-7941-4707-8a4e-4483165111b6 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:58.644296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1069/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:06:58.679819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab9482-b198-4791-a718-0a7f503e3a0a service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:58.681572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5fab9482-b198-4791-a718-0a7f503e3a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:58.698840 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab9482-b198-4791-a718-0a7f503e3a0a service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:58.703714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:58.704648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.704965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.705242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] PATH_INFO: `/v3/users/79e64aa72b564a87bcf5013a6693dc61` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.706489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e64aa72b564a87bcf5013a6693dc61)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:58.709920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:58.789729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e64aa72b564a87bcf5013a6693dc61'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:58.790483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e64aa72b564a87bcf5013a6693dc61'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:58.797866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e64aa72b564a87bcf5013a6693dc61'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:58.798654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e64aa72b564a87bcf5013a6693dc61'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:58.808268 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0725e747-b556-4b58-8f96-7d3e904057eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e64aa72b564a87bcf5013a6693dc61'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:58.828261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1063/2132] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:58 2022] DELETE /identity/v3/users/79e64aa72b564a87bcf5013a6693dc61 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:58.833281 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846caf09-297b-4647-a424-623f91652c40 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.833690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846caf09-297b-4647-a424-623f91652c40 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.834219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846caf09-297b-4647-a424-623f91652c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.854133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-846caf09-297b-4647-a424-623f91652c40 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:58.855476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-846caf09-297b-4647-a424-623f91652c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:58.887591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077aad5-f439-4c5b-8622-66a82a84e964 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.887985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077aad5-f439-4c5b-8622-66a82a84e964 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.888368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077aad5-f439-4c5b-8622-66a82a84e964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.948704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1070/2133] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:58.952210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846caf09-297b-4647-a424-623f91652c40 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:58.953559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c077aad5-f439-4c5b-8622-66a82a84e964 None None] MFA Rules not processed for user `cc19addacb9c412cb008e243c1fc7ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:58.955222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c077aad5-f439-4c5b-8622-66a82a84e964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:58.959634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-846caf09-297b-4647-a424-623f91652c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:58.978076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846caf09-297b-4647-a424-623f91652c40 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:58.984571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:58.985443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:58.985725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:58.986002 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] PATH_INFO: `/v3/projects/ec672910df094df5927552f16155841b` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:58.987207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec672910df094df5927552f16155841b)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:58.989940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.027166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:59.047069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec672910df094df5927552f16155841b', 'request_id': 'req-1879998e-6a29-4f8c-9146-e2805c796849'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.055226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec672910df094df5927552f16155841b', 'request_id': 'req-1879998e-6a29-4f8c-9146-e2805c796849'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.059694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1064/2134] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:59.059694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077aad5-f439-4c5b-8622-66a82a84e964 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.061528 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c077aad5-f439-4c5b-8622-66a82a84e964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:59.070796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec672910df094df5927552f16155841b', 'request_id': 'req-1879998e-6a29-4f8c-9146-e2805c796849'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.096743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] The token cache is being invalidate because project ec672910df094df5927552f16155841b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:59.097079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.097929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.100567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1071/2135] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:58 2022] DELETE /identity/v3/projects/ec672910df094df5927552f16155841b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:59.101464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.102762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:59.139969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:59.146876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077aad5-f439-4c5b-8622-66a82a84e964 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:59.174812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879998e-6a29-4f8c-9146-e2805c796849 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:59.179723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437de99f-d35b-4e1c-880e-aaba58c86394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:59.181054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437de99f-d35b-4e1c-880e-aaba58c86394 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.181365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437de99f-d35b-4e1c-880e-aaba58c86394 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.181963 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437de99f-d35b-4e1c-880e-aaba58c86394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.183823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-437de99f-d35b-4e1c-880e-aaba58c86394 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:59.206507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651f5ba-dcb4-4073-9f8e-89430e0b99e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:59.207149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651f5ba-dcb4-4073-9f8e-89430e0b99e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.207698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651f5ba-dcb4-4073-9f8e-89430e0b99e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.208115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651f5ba-dcb4-4073-9f8e-89430e0b99e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.209970 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5651f5ba-dcb4-4073-9f8e-89430e0b99e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:59.254890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437de99f-d35b-4e1c-880e-aaba58c86394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:59.256351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437de99f-d35b-4e1c-880e-aaba58c86394 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.265819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1072/2136] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:59.267080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0c50-d9c6-49bf-b1f0-51a53803143d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.267378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0c50-d9c6-49bf-b1f0-51a53803143d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.267705 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0c50-d9c6-49bf-b1f0-51a53803143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.281873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651f5ba-dcb4-4073-9f8e-89430e0b99e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:59.283605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651f5ba-dcb4-4073-9f8e-89430e0b99e4 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.287465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-1b9b0c50-d9c6-49bf-b1f0-51a53803143d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:59.288706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1b9b0c50-d9c6-49bf-b1f0-51a53803143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:59.296229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1065/2137] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:06:59.296827 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1073/2138] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:06:59.304723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4129aa-6220-42aa-9f0b-8ea352a87dd2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.305085 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4129aa-6220-42aa-9f0b-8ea352a87dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.305435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4129aa-6220-42aa-9f0b-8ea352a87dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.326403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1e4129aa-6220-42aa-9f0b-8ea352a87dd2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:59.327707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1e4129aa-6220-42aa-9f0b-8ea352a87dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:59.337385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1066/2139] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:59.341865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b0c50-d9c6-49bf-b1f0-51a53803143d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.354417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b0c50-d9c6-49bf-b1f0-51a53803143d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:59.361484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c76ea6-4965-46f2-a897-1953b46f1e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:59.362557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c76ea6-4965-46f2-a897-1953b46f1e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.362704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c76ea6-4965-46f2-a897-1953b46f1e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.362952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c76ea6-4965-46f2-a897-1953b46f1e8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.363933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c76ea6-4965-46f2-a897-1953b46f1e8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:06:59.364103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c76ea6-4965-46f2-a897-1953b46f1e8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:59.365860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c76ea6-4965-46f2-a897-1953b46f1e8a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.382193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1074/2140] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 02:06:59.387904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4129aa-6220-42aa-9f0b-8ea352a87dd2 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.398883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4129aa-6220-42aa-9f0b-8ea352a87dd2 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:59.408284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414551e-113a-4200-8a6e-610872724db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:59.409070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414551e-113a-4200-8a6e-610872724db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.409356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414551e-113a-4200-8a6e-610872724db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.409686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414551e-113a-4200-8a6e-610872724db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.410727 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414551e-113a-4200-8a6e-610872724db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:59.414084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414551e-113a-4200-8a6e-610872724db3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.458854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1067/2141] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:06:59 2022] POST /identity/v3/projects => generated 369 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:59.459739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c76ea6-4965-46f2-a897-1953b46f1e8a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.477727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c76ea6-4965-46f2-a897-1953b46f1e8a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:59.557386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:59.582130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c011ac-dafd-4079-8a0f-8a0a93998f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:59.583855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c011ac-dafd-4079-8a0f-8a0a93998f13 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.584641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c011ac-dafd-4079-8a0f-8a0a93998f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.585085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c011ac-dafd-4079-8a0f-8a0a93998f13 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.585871 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c011ac-dafd-4079-8a0f-8a0a93998f13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:59.588037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c011ac-dafd-4079-8a0f-8a0a93998f13 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.623414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1075/2142] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:06:59 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:06:59.624657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414551e-113a-4200-8a6e-610872724db3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.633405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c011ac-dafd-4079-8a0f-8a0a93998f13 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.635876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-58c011ac-dafd-4079-8a0f-8a0a93998f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:59.643623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414551e-113a-4200-8a6e-610872724db3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:59.648768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c011ac-dafd-4079-8a0f-8a0a93998f13 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:59.651924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ecbe3-a1bb-438e-a4bd-79fb288fd3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:59.652926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ecbe3-a1bb-438e-a4bd-79fb288fd3dc admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.653214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ecbe3-a1bb-438e-a4bd-79fb288fd3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.653491 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ecbe3-a1bb-438e-a4bd-79fb288fd3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.654557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ecbe3-a1bb-438e-a4bd-79fb288fd3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:59.655967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ecbe3-a1bb-438e-a4bd-79fb288fd3dc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.658444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:59.659575 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.660236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.660666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] PATH_INFO: `/v3/projects/4cfe116cc6654d12b70df88a3489bd5e` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.662189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cfe116cc6654d12b70df88a3489bd5e)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:59.664594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.673033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1068/2143] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:06:59.677704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ecbe3-a1bb-438e-a4bd-79fb288fd3dc admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.691120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ecbe3-a1bb-438e-a4bd-79fb288fd3dc admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:59.699832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31172e4-5589-4fa6-8c0f-482c39df4427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:59.700676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31172e4-5589-4fa6-8c0f-482c39df4427 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.701028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31172e4-5589-4fa6-8c0f-482c39df4427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.701460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31172e4-5589-4fa6-8c0f-482c39df4427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.702346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31172e4-5589-4fa6-8c0f-482c39df4427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:59.704224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31172e4-5589-4fa6-8c0f-482c39df4427 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.715739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1069/2144] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:06:59.719845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31172e4-5589-4fa6-8c0f-482c39df4427 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.722604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cfe116cc6654d12b70df88a3489bd5e', 'request_id': 'req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.731911 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cfe116cc6654d12b70df88a3489bd5e', 'request_id': 'req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.733947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31172e4-5589-4fa6-8c0f-482c39df4427 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:59.739033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cfe116cc6654d12b70df88a3489bd5e', 'request_id': 'req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.753234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] The token cache is being invalidate because project 4cfe116cc6654d12b70df88a3489bd5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:06:59.753923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.754643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.756470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1076/2145] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:06:59 2022] DELETE /identity/v3/projects/4cfe116cc6654d12b70df88a3489bd5e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:06:59.760330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.761717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:59.792728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198e06e-c6ed-4484-942e-bc11d2bc606e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:59.793628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198e06e-c6ed-4484-942e-bc11d2bc606e admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.793992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198e06e-c6ed-4484-942e-bc11d2bc606e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.794167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198e06e-c6ed-4484-942e-bc11d2bc606e admin admin] PATH_INFO: `/v3/projects/78323d93142d481f812c5108ce58d5f1/users/afce14721a694cafbba342b6ef76b635/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.798521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:59.813446 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198e06e-c6ed-4484-942e-bc11d2bc606e admin admin] RBAC: Authorizing `identity:create_grant(project_id=78323d93142d481f812c5108ce58d5f1, user_id=afce14721a694cafbba342b6ef76b635, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:59.817019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198e06e-c6ed-4484-942e-bc11d2bc606e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.830156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1070/2146] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:06:59 2022] PUT /identity/v3/projects/78323d93142d481f812c5108ce58d5f1/users/afce14721a694cafbba342b6ef76b635/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:06:59.838883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c606eb-ef99-4574-8285-b3b5ee9bc9c0 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:06:59.841914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5da1f-16dd-47d0-b3d7-d752e227446f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.842207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5da1f-16dd-47d0-b3d7-d752e227446f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.842612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5da1f-16dd-47d0-b3d7-d752e227446f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.869123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d7a5da1f-16dd-47d0-b3d7-d752e227446f None None] MFA Rules not processed for user `afce14721a694cafbba342b6ef76b635`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:06:59.870383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d7a5da1f-16dd-47d0-b3d7-d752e227446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:59.899472 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:06:59.900864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:06:59.901157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:06:59.901438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] PATH_INFO: `/v3/users/12c6a3c5b64c49ffac9893fde0855184` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:06:59.902844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] RBAC: Authorizing `identity:delete_user(user_id=12c6a3c5b64c49ffac9893fde0855184)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:06:59.905552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:06:59.925720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:06:59.946060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1071/2147] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:06:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:06:59.959787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5da1f-16dd-47d0-b3d7-d752e227446f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:06:59.961833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d7a5da1f-16dd-47d0-b3d7-d752e227446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:06:59.987364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12c6a3c5b64c49ffac9893fde0855184'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.988100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12c6a3c5b64c49ffac9893fde0855184'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.999105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12c6a3c5b64c49ffac9893fde0855184'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:06:59.999851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12c6a3c5b64c49ffac9893fde0855184'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:00.014624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12c6a3c5b64c49ffac9893fde0855184'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:00.030210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1077/2148] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:06:59 2022] DELETE /identity/v3/users/12c6a3c5b64c49ffac9893fde0855184 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:00.059269 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5da1f-16dd-47d0-b3d7-d752e227446f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:00.124342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937c37c-597f-4eb5-ad07-e76395233011 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d1e01f6f82a4ebbb15d0cb55b963af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:00.125543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937c37c-597f-4eb5-ad07-e76395233011 service glance] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:00.125907 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937c37c-597f-4eb5-ad07-e76395233011 service glance] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:00.126259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937c37c-597f-4eb5-ad07-e76395233011 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:00.128651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3937c37c-597f-4eb5-ad07-e76395233011 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:00.185696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:00.214590 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937c37c-597f-4eb5-ad07-e76395233011 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:00.216075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937c37c-597f-4eb5-ad07-e76395233011 service glance] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:00.227247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1072/2149] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:06:59 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:07:00.419314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:00.420895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:00.520627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8d769-e20e-49cd-82d0-4d8c6eb2a35b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:00.525782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:00.527352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:00.527352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:00.527611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] PATH_INFO: `/v3/projects/bde984819d4a40e38d9554878fac56a9` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:00.529938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde984819d4a40e38d9554878fac56a9)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:00.531450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:00.609804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde984819d4a40e38d9554878fac56a9', 'request_id': 'req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:00.617833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde984819d4a40e38d9554878fac56a9', 'request_id': 'req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:00.631575 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde984819d4a40e38d9554878fac56a9', 'request_id': 'req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:00.653236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] The token cache is being invalidate because project bde984819d4a40e38d9554878fac56a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:00.653539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:00.655735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-42ce574b-a0db-4ad1-bb4d-621b38ab86d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:00.657430 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1078/2150] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:00 2022] DELETE /identity/v3/projects/bde984819d4a40e38d9554878fac56a9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:00.666177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3561891-aecf-4dcd-96c7-d93aaffa059b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:00.666475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3561891-aecf-4dcd-96c7-d93aaffa059b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:00.666756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3561891-aecf-4dcd-96c7-d93aaffa059b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:00.710987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e3561891-aecf-4dcd-96c7-d93aaffa059b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:00.712626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e3561891-aecf-4dcd-96c7-d93aaffa059b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:00.748117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601324fb-6e3e-4494-bb8f-a408ea0eeac8 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:00.748644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601324fb-6e3e-4494-bb8f-a408ea0eeac8 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:00.749085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601324fb-6e3e-4494-bb8f-a408ea0eeac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:00.775822 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-601324fb-6e3e-4494-bb8f-a408ea0eeac8 None None] MFA Rules not processed for user `cc19addacb9c412cb008e243c1fc7ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:00.777049 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-601324fb-6e3e-4494-bb8f-a408ea0eeac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:00.799034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:00.834841 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1073/2151] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:00.849318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cbcbd-dfba-4a1b-9624-1957d98ef5a6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:00.849318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cbcbd-dfba-4a1b-9624-1957d98ef5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:00.849318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cbcbd-dfba-4a1b-9624-1957d98ef5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:00.873099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fd8cbcbd-dfba-4a1b-9624-1957d98ef5a6 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:00.874862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fd8cbcbd-dfba-4a1b-9624-1957d98ef5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:00.887176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1074/2152] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:07:00.892234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8cbcbd-dfba-4a1b-9624-1957d98ef5a6 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:00.903926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8cbcbd-dfba-4a1b-9624-1957d98ef5a6 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:00.922279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1079/2153] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:00.927316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db155ab2-f993-4683-a9c8-ec4589befd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:00.928075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db155ab2-f993-4683-a9c8-ec4589befd7d admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:00.928566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db155ab2-f993-4683-a9c8-ec4589befd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:00.928994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db155ab2-f993-4683-a9c8-ec4589befd7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:00.929939 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db155ab2-f993-4683-a9c8-ec4589befd7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:07:00.930252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db155ab2-f993-4683-a9c8-ec4589befd7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:00.931899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db155ab2-f993-4683-a9c8-ec4589befd7d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:00.947843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1075/2154] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:07:00.951865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601324fb-6e3e-4494-bb8f-a408ea0eeac8 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:00.972325 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601324fb-6e3e-4494-bb8f-a408ea0eeac8 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:00.994718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc16176-c971-42ee-8b96-afc3b7d7f5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:00.994718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc16176-c971-42ee-8b96-afc3b7d7f5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:00.994718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc16176-c971-42ee-8b96-afc3b7d7f5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:00.994718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc16176-c971-42ee-8b96-afc3b7d7f5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:00.995079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc16176-c971-42ee-8b96-afc3b7d7f5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:00.998564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc16176-c971-42ee-8b96-afc3b7d7f5e7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:01.056441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1080/2155] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:00 2022] POST /identity/v3/projects => generated 409 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:07:01.060208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db155ab2-f993-4683-a9c8-ec4589befd7d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:01.077277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db155ab2-f993-4683-a9c8-ec4589befd7d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:01.152891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7827fd-1ee7-4951-89de-a75d88b08dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:01.154406 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7827fd-1ee7-4951-89de-a75d88b08dda admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:01.154705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7827fd-1ee7-4951-89de-a75d88b08dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:01.154987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7827fd-1ee7-4951-89de-a75d88b08dda admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:01.155694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7827fd-1ee7-4951-89de-a75d88b08dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:01.157912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7827fd-1ee7-4951-89de-a75d88b08dda admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:01.183269 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1076/2156] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:01 2022] POST /identity/v3/users => generated 305 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:01.187331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc16176-c971-42ee-8b96-afc3b7d7f5e7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:01.199020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc16176-c971-42ee-8b96-afc3b7d7f5e7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:01.205935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832cd5e-e882-461d-852d-e14b8bd542b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:01.206768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832cd5e-e882-461d-852d-e14b8bd542b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:01.207019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832cd5e-e882-461d-852d-e14b8bd542b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:01.207905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832cd5e-e882-461d-852d-e14b8bd542b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:01.208880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832cd5e-e882-461d-852d-e14b8bd542b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:01.210763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832cd5e-e882-461d-852d-e14b8bd542b1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:01.223604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1081/2157] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:07:01.228812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7827fd-1ee7-4951-89de-a75d88b08dda admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:01.243747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7827fd-1ee7-4951-89de-a75d88b08dda admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:01.255913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a1484-e76a-4733-b571-036b1dad913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:01.257064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a1484-e76a-4733-b571-036b1dad913e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:01.257445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a1484-e76a-4733-b571-036b1dad913e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:01.257910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a1484-e76a-4733-b571-036b1dad913e admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:01.258745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a1484-e76a-4733-b571-036b1dad913e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:01.260961 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a1484-e76a-4733-b571-036b1dad913e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:01.284126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1077/2158] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:01.288912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832cd5e-e882-461d-852d-e14b8bd542b1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:01.308023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832cd5e-e882-461d-852d-e14b8bd542b1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:01.313605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772df36-4c58-4e00-bcd7-58eb7f145a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:01.314660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772df36-4c58-4e00-bcd7-58eb7f145a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:01.315011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772df36-4c58-4e00-bcd7-58eb7f145a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:01.315362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772df36-4c58-4e00-bcd7-58eb7f145a94 admin admin] PATH_INFO: `/v3/projects/d641a85e873144a981b4c1fc60c17db3/users/927b174230834bb585671281d6305b3d/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:01.359602 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772df36-4c58-4e00-bcd7-58eb7f145a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d641a85e873144a981b4c1fc60c17db3, user_id=927b174230834bb585671281d6305b3d, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:01.362336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772df36-4c58-4e00-bcd7-58eb7f145a94 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:01.388613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1082/2159] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:01 2022] PUT /identity/v3/projects/d641a85e873144a981b4c1fc60c17db3/users/927b174230834bb585671281d6305b3d/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:01.403597 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42d91-5723-441a-8d75-5dd9e25f5009 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:01.404083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42d91-5723-441a-8d75-5dd9e25f5009 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:01.404317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42d91-5723-441a-8d75-5dd9e25f5009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:01.453835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-49a42d91-5723-441a-8d75-5dd9e25f5009 None None] MFA Rules not processed for user `927b174230834bb585671281d6305b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:01.455328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-49a42d91-5723-441a-8d75-5dd9e25f5009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:01.513605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:01.586880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1078/2160] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:01.587621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772df36-4c58-4e00-bcd7-58eb7f145a94 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:01.695188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772df36-4c58-4e00-bcd7-58eb7f145a94 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:01.701010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8c1d7-4e9b-4412-94f4-5986ae0a604b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:01.701901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8c1d7-4e9b-4412-94f4-5986ae0a604b admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:01.702320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8c1d7-4e9b-4412-94f4-5986ae0a604b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:01.703051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8c1d7-4e9b-4412-94f4-5986ae0a604b admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:01.703911 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8c1d7-4e9b-4412-94f4-5986ae0a604b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:01.706242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8c1d7-4e9b-4412-94f4-5986ae0a604b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:01.754560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1083/2161] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:01 2022] POST /identity/v3/projects => generated 407 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:01.758678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a42d91-5723-441a-8d75-5dd9e25f5009 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:01.774287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a42d91-5723-441a-8d75-5dd9e25f5009 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:01.873634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b835553-69f1-448c-b41a-213167c7f49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:01.877929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b835553-69f1-448c-b41a-213167c7f49f admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:01.878356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b835553-69f1-448c-b41a-213167c7f49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:01.878785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b835553-69f1-448c-b41a-213167c7f49f admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:01.880225 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b835553-69f1-448c-b41a-213167c7f49f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:01.883109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b835553-69f1-448c-b41a-213167c7f49f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:01.947218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1079/2162] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:01 2022] POST /identity/v3/users => generated 310 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:01.952473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8c1d7-4e9b-4412-94f4-5986ae0a604b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:01.982024 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8c1d7-4e9b-4412-94f4-5986ae0a604b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:01.999868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55539499-0de4-420f-a415-f3fe64e22ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:02.000733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55539499-0de4-420f-a415-f3fe64e22ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:02.000899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55539499-0de4-420f-a415-f3fe64e22ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:02.002594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55539499-0de4-420f-a415-f3fe64e22ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:02.002594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55539499-0de4-420f-a415-f3fe64e22ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:02.004068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55539499-0de4-420f-a415-f3fe64e22ac8 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:02.044441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1084/2163] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:07:02.065379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b835553-69f1-448c-b41a-213167c7f49f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:02.074425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b835553-69f1-448c-b41a-213167c7f49f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:02.079884 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d33f9-c1b6-4844-bf5a-9724ea0410f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:02.080986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d33f9-c1b6-4844-bf5a-9724ea0410f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:02.081328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d33f9-c1b6-4844-bf5a-9724ea0410f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:02.081679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d33f9-c1b6-4844-bf5a-9724ea0410f1 admin admin] PATH_INFO: `/v3/projects/ee4fafbcccc24d9aad00975656b00e04/users/2794940b4d8542c888c30ece8c471aee/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:02.109866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d33f9-c1b6-4844-bf5a-9724ea0410f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4fafbcccc24d9aad00975656b00e04, user_id=2794940b4d8542c888c30ece8c471aee, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:02.113870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d33f9-c1b6-4844-bf5a-9724ea0410f1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:02.141104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1080/2164] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:02 2022] PUT /identity/v3/projects/ee4fafbcccc24d9aad00975656b00e04/users/2794940b4d8542c888c30ece8c471aee/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:02.142805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55539499-0de4-420f-a415-f3fe64e22ac8 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:02.168366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55539499-0de4-420f-a415-f3fe64e22ac8 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:02.347182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:02.396955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713a613-e83d-4540-81c5-b252c8216075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:02.397844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713a613-e83d-4540-81c5-b252c8216075 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:02.398133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713a613-e83d-4540-81c5-b252c8216075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:02.398417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713a613-e83d-4540-81c5-b252c8216075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:02.399214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713a613-e83d-4540-81c5-b252c8216075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:02.400886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713a613-e83d-4540-81c5-b252c8216075 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:02.442229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1085/2165] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:02.444072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d33f9-c1b6-4844-bf5a-9724ea0410f1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:02.460065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d33f9-c1b6-4844-bf5a-9724ea0410f1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:02.473373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30eaa96-ce6c-4dca-8224-07a74b035e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:02.476911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eaa96-ce6c-4dca-8224-07a74b035e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:02.479544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eaa96-ce6c-4dca-8224-07a74b035e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:02.480418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eaa96-ce6c-4dca-8224-07a74b035e32 admin admin] PATH_INFO: `/v3/projects/ee4fafbcccc24d9aad00975656b00e04/users/2794940b4d8542c888c30ece8c471aee/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:02.489572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30eaa96-ce6c-4dca-8224-07a74b035e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4fafbcccc24d9aad00975656b00e04, user_id=2794940b4d8542c888c30ece8c471aee, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:02.493236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30eaa96-ce6c-4dca-8224-07a74b035e32 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:02.508439 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1081/2166] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:02 2022] PUT /identity/v3/projects/ee4fafbcccc24d9aad00975656b00e04/users/2794940b4d8542c888c30ece8c471aee/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:02.519199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8381a-aa2e-4a13-b148-20fe02a7d66a None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:02.519627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8381a-aa2e-4a13-b148-20fe02a7d66a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:02.520348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8381a-aa2e-4a13-b148-20fe02a7d66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:02.580828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-75c8381a-aa2e-4a13-b148-20fe02a7d66a None None] MFA Rules not processed for user `2794940b4d8542c888c30ece8c471aee`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:02.583332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-75c8381a-aa2e-4a13-b148-20fe02a7d66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:02.726918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1086/2167] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:02 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:02.745099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30eaa96-ce6c-4dca-8224-07a74b035e32 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:02.746851 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a30eaa96-ce6c-4dca-8224-07a74b035e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:02.831237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30eaa96-ce6c-4dca-8224-07a74b035e32 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:02.835844 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe5525-ac76-4257-861f-cb16a336b2d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:02.836626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe5525-ac76-4257-861f-cb16a336b2d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:02.836878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe5525-ac76-4257-861f-cb16a336b2d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:02.837384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe5525-ac76-4257-861f-cb16a336b2d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:02.851711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe5525-ac76-4257-861f-cb16a336b2d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:02.851711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe5525-ac76-4257-861f-cb16a336b2d3 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:02.863853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1082/2168] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:02 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:03.203784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8381a-aa2e-4a13-b148-20fe02a7d66a None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:03.214057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8381a-aa2e-4a13-b148-20fe02a7d66a None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:03.217736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8213c72-47a7-40a1-aea2-8f05c52d1e68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:03.218437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8213c72-47a7-40a1-aea2-8f05c52d1e68 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:03.218733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8213c72-47a7-40a1-aea2-8f05c52d1e68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:03.219011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8213c72-47a7-40a1-aea2-8f05c52d1e68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:03.276738 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8213c72-47a7-40a1-aea2-8f05c52d1e68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:03.278331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8213c72-47a7-40a1-aea2-8f05c52d1e68 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:03.288004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1087/2169] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:03.463125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe5525-ac76-4257-861f-cb16a336b2d3 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:03.473889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cdbe5525-ac76-4257-861f-cb16a336b2d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:03.556116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:03.610611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe5525-ac76-4257-861f-cb16a336b2d3 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:03.615738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:03.616737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:03.617022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:03.617296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] PATH_INFO: `/v3/users/a30aec487b5a439a80962f9f76e93c4a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:03.618675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a30aec487b5a439a80962f9f76e93c4a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:03.620636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:03.693159 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a30aec487b5a439a80962f9f76e93c4a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:03.695932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a30aec487b5a439a80962f9f76e93c4a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:03.705153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a30aec487b5a439a80962f9f76e93c4a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:03.705975 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a30aec487b5a439a80962f9f76e93c4a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:03.715814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a30aec487b5a439a80962f9f76e93c4a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:03.736206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1083/2170] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:03 2022] DELETE /identity/v3/users/a30aec487b5a439a80962f9f76e93c4a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:03.740808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e4bcd-cc60-4a5e-999d-ac624add4166 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:03.741396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e4bcd-cc60-4a5e-999d-ac624add4166 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:03.741396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e4bcd-cc60-4a5e-999d-ac624add4166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:03.759531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ea3e4bcd-cc60-4a5e-999d-ac624add4166 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:03.760773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ea3e4bcd-cc60-4a5e-999d-ac624add4166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:03.820124 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1088/2171] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:03.829691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:03.831136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:03.891845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be08f3-b0a4-44bf-9c97-a4a9ddf2e0c7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:03.896309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd6e69-d1a3-4331-97f1-8e04b414579f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:03.896997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd6e69-d1a3-4331-97f1-8e04b414579f service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:03.897284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd6e69-d1a3-4331-97f1-8e04b414579f service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:03.897564 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd6e69-d1a3-4331-97f1-8e04b414579f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:03.906820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd6e69-d1a3-4331-97f1-8e04b414579f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:03.908284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd6e69-d1a3-4331-97f1-8e04b414579f service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:03.920261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1084/2172] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:04.119111 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e4bcd-cc60-4a5e-999d-ac624add4166 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:04.122500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ea3e4bcd-cc60-4a5e-999d-ac624add4166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:04.131165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:04.137671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e4bcd-cc60-4a5e-999d-ac624add4166 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:04.150155 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:04.151453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:04.151868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:04.152539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] PATH_INFO: `/v3/users/9529ff7fa5594d5090a0a2cdd8f4d42f` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:04.154022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9529ff7fa5594d5090a0a2cdd8f4d42f)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:04.157064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:04.247870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9529ff7fa5594d5090a0a2cdd8f4d42f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:04.248817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9529ff7fa5594d5090a0a2cdd8f4d42f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:04.257541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9529ff7fa5594d5090a0a2cdd8f4d42f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:04.258370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9529ff7fa5594d5090a0a2cdd8f4d42f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:04.284236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16f23822-f803-4114-a470-7cca5f0a2353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9529ff7fa5594d5090a0a2cdd8f4d42f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:04.287219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd6e69-d1a3-4331-97f1-8e04b414579f service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:04.289344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a9bd6e69-d1a3-4331-97f1-8e04b414579f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:04.303614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1089/2173] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:04 2022] DELETE /identity/v3/users/9529ff7fa5594d5090a0a2cdd8f4d42f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:04.305203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09e120-60bd-4bf2-b707-64b3399c25c7 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:04.305479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09e120-60bd-4bf2-b707-64b3399c25c7 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:04.305760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09e120-60bd-4bf2-b707-64b3399c25c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:04.333843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd6e69-d1a3-4331-97f1-8e04b414579f service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:04.342889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7c09e120-60bd-4bf2-b707-64b3399c25c7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:04.344120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7c09e120-60bd-4bf2-b707-64b3399c25c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:04.387547 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:04.402909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:04.404066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:04.404697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:04.405288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] PATH_INFO: `/v3/projects/4318ce9be18545dfabe276ed4a935a5e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:04.407135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4318ce9be18545dfabe276ed4a935a5e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:04.409340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:04.425691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1090/2174] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:04.435603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09e120-60bd-4bf2-b707-64b3399c25c7 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:04.437878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7c09e120-60bd-4bf2-b707-64b3399c25c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:04.472094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4318ce9be18545dfabe276ed4a935a5e', 'request_id': 'req-240cf1b1-528a-4581-9028-2a1d530a8e0b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:04.482679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4318ce9be18545dfabe276ed4a935a5e', 'request_id': 'req-240cf1b1-528a-4581-9028-2a1d530a8e0b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:04.492573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4318ce9be18545dfabe276ed4a935a5e', 'request_id': 'req-240cf1b1-528a-4581-9028-2a1d530a8e0b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:04.507269 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] The token cache is being invalidate because project 4318ce9be18545dfabe276ed4a935a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:04.508952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:04.509066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09e120-60bd-4bf2-b707-64b3399c25c7 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:04.509984 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:04.514713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1085/2175] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:04 2022] DELETE /identity/v3/projects/4318ce9be18545dfabe276ed4a935a5e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:04.624149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:04.692874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90268ab8-3e35-4858-9d9e-7567a3c55944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:04.692874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90268ab8-3e35-4858-9d9e-7567a3c55944 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:04.693120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90268ab8-3e35-4858-9d9e-7567a3c55944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:04.694197 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90268ab8-3e35-4858-9d9e-7567a3c55944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:04.696078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-90268ab8-3e35-4858-9d9e-7567a3c55944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:04.810877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90268ab8-3e35-4858-9d9e-7567a3c55944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:04.812444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90268ab8-3e35-4858-9d9e-7567a3c55944 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:04.835196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1091/2176] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:07:05.573531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:05.575132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:05.593342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cf1b1-528a-4581-9028-2a1d530a8e0b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:05.604343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:05.605271 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:05.605557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:05.605833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] PATH_INFO: `/v3/projects/5a3a985d89094bf991e25441494878b0` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:05.607075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a3a985d89094bf991e25441494878b0)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:05.609120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:05.660197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a3a985d89094bf991e25441494878b0', 'request_id': 'req-4becdb34-2372-4fdd-a589-d13b2991d52f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:05.666625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a3a985d89094bf991e25441494878b0', 'request_id': 'req-4becdb34-2372-4fdd-a589-d13b2991d52f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:05.673652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a3a985d89094bf991e25441494878b0', 'request_id': 'req-4becdb34-2372-4fdd-a589-d13b2991d52f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:05.685509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] The token cache is being invalidate because project 5a3a985d89094bf991e25441494878b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:05.685806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:05.686463 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:05.689091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1086/2177] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:05 2022] DELETE /identity/v3/projects/5a3a985d89094bf991e25441494878b0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:05.693744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90268ab8-3e35-4858-9d9e-7567a3c55944 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:05.704387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90268ab8-3e35-4858-9d9e-7567a3c55944 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:05.761556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:05.762424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:05.762704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:05.762978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] PATH_INFO: `/v3/users/0b11fba371e1436392ba7bab75de1b81` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:05.764149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b11fba371e1436392ba7bab75de1b81)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:05.766121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:05.818019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b11fba371e1436392ba7bab75de1b81'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:05.818680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b11fba371e1436392ba7bab75de1b81'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:05.825522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b11fba371e1436392ba7bab75de1b81'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:05.826168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b11fba371e1436392ba7bab75de1b81'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:05.838573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-754d89ca-99c8-41e2-89af-831482d59917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b11fba371e1436392ba7bab75de1b81'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:05.849732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:05.855062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1092/2178] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:05 2022] DELETE /identity/v3/users/0b11fba371e1436392ba7bab75de1b81 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:06.233725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:06.235312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:06.249306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4becdb34-2372-4fdd-a589-d13b2991d52f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:06.311392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:06.312265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:06.312585 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:06.312863 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] PATH_INFO: `/v3/projects/cb6eb940d0e74578ba139dc19b3a5dea` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:06.314125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6eb940d0e74578ba139dc19b3a5dea)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:06.316005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:06.322615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:06.367285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6eb940d0e74578ba139dc19b3a5dea', 'request_id': 'req-97744a89-957a-4cd7-8259-8ae2f99dcdc7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:06.373273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6eb940d0e74578ba139dc19b3a5dea', 'request_id': 'req-97744a89-957a-4cd7-8259-8ae2f99dcdc7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:06.382212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6eb940d0e74578ba139dc19b3a5dea', 'request_id': 'req-97744a89-957a-4cd7-8259-8ae2f99dcdc7'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:06.394221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] The token cache is being invalidate because project cb6eb940d0e74578ba139dc19b3a5dea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:06.395460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:06.396264 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-97744a89-957a-4cd7-8259-8ae2f99dcdc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:06.398017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1087/2179] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:06 2022] DELETE /identity/v3/projects/cb6eb940d0e74578ba139dc19b3a5dea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:06.403470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8150e-4ea0-4bb9-b0ac-fbc71cbbe554 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:06.403775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8150e-4ea0-4bb9-b0ac-fbc71cbbe554 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:06.404040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8150e-4ea0-4bb9-b0ac-fbc71cbbe554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:06.423770 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-9eb8150e-4ea0-4bb9-b0ac-fbc71cbbe554 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:06.425121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-9eb8150e-4ea0-4bb9-b0ac-fbc71cbbe554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:06.487573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1093/2180] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:06.496325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17aed5-1006-4bd6-a61d-ec85f8567229 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:06.496767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17aed5-1006-4bd6-a61d-ec85f8567229 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:06.497135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17aed5-1006-4bd6-a61d-ec85f8567229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:06.515813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5e17aed5-1006-4bd6-a61d-ec85f8567229 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:06.517051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5e17aed5-1006-4bd6-a61d-ec85f8567229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:06.526354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1088/2181] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:07:06.530715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8150e-4ea0-4bb9-b0ac-fbc71cbbe554 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:06.540949 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8150e-4ea0-4bb9-b0ac-fbc71cbbe554 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:06.545767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bbe38-a964-4bdd-9f3f-d587a331019e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:06.546453 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bbe38-a964-4bdd-9f3f-d587a331019e admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:06.546742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bbe38-a964-4bdd-9f3f-d587a331019e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:06.547023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bbe38-a964-4bdd-9f3f-d587a331019e admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:06.547834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bbe38-a964-4bdd-9f3f-d587a331019e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:07:06.548103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bbe38-a964-4bdd-9f3f-d587a331019e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:06.549780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bbe38-a964-4bdd-9f3f-d587a331019e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:06.564903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1094/2182] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:07:06.569964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17aed5-1006-4bd6-a61d-ec85f8567229 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:06.579726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17aed5-1006-4bd6-a61d-ec85f8567229 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:06.584493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31de652-2556-4ed0-8b39-8d4ed1df1cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:06.585144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31de652-2556-4ed0-8b39-8d4ed1df1cee admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:06.585424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31de652-2556-4ed0-8b39-8d4ed1df1cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:06.585706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31de652-2556-4ed0-8b39-8d4ed1df1cee admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:06.586378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31de652-2556-4ed0-8b39-8d4ed1df1cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:06.588266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31de652-2556-4ed0-8b39-8d4ed1df1cee admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:06.613825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1089/2183] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:06 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:06.618011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bbe38-a964-4bdd-9f3f-d587a331019e admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:06.627947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bbe38-a964-4bdd-9f3f-d587a331019e admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:06.690192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e976e9-ac48-4fc3-9a74-7366735c5ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:06.690905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e976e9-ac48-4fc3-9a74-7366735c5ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:06.691195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e976e9-ac48-4fc3-9a74-7366735c5ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:06.691511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e976e9-ac48-4fc3-9a74-7366735c5ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:06.692355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e976e9-ac48-4fc3-9a74-7366735c5ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:06.694329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e976e9-ac48-4fc3-9a74-7366735c5ad3 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:06.715536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1095/2184] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:06 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:06.719990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31de652-2556-4ed0-8b39-8d4ed1df1cee admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:06.731825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31de652-2556-4ed0-8b39-8d4ed1df1cee admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:06.737819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fa534-dbcd-4998-99eb-21a009c1f495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:06.739148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fa534-dbcd-4998-99eb-21a009c1f495 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:06.739437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fa534-dbcd-4998-99eb-21a009c1f495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:06.739715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fa534-dbcd-4998-99eb-21a009c1f495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:06.740480 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fa534-dbcd-4998-99eb-21a009c1f495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:06.742128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fa534-dbcd-4998-99eb-21a009c1f495 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:06.752889 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1090/2185] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:06.758140 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e976e9-ac48-4fc3-9a74-7366735c5ad3 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:06.768524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e976e9-ac48-4fc3-9a74-7366735c5ad3 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:06.773846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88dee3-ba24-4e00-8e6d-0d47f5bd741f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:06.774503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88dee3-ba24-4e00-8e6d-0d47f5bd741f admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:06.774785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88dee3-ba24-4e00-8e6d-0d47f5bd741f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:06.775083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88dee3-ba24-4e00-8e6d-0d47f5bd741f admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:06.775809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88dee3-ba24-4e00-8e6d-0d47f5bd741f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:06.777437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88dee3-ba24-4e00-8e6d-0d47f5bd741f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:06.788523 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1096/2186] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:06.792603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fa534-dbcd-4998-99eb-21a009c1f495 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:06.804116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fa534-dbcd-4998-99eb-21a009c1f495 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:06.808762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e750a01-a30f-4881-9357-1accd58bf92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:06.809634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e750a01-a30f-4881-9357-1accd58bf92f admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:06.809914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e750a01-a30f-4881-9357-1accd58bf92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:06.810185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e750a01-a30f-4881-9357-1accd58bf92f admin admin] PATH_INFO: `/v3/projects/a9c252fa9af94e0c90881766e2925b60/users/a76dc55234bd4df1aa8de8156136ff64/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:06.829576 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e750a01-a30f-4881-9357-1accd58bf92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c252fa9af94e0c90881766e2925b60, user_id=a76dc55234bd4df1aa8de8156136ff64, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:06.833384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e750a01-a30f-4881-9357-1accd58bf92f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:06.850558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1091/2187] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:06 2022] PUT /identity/v3/projects/a9c252fa9af94e0c90881766e2925b60/users/a76dc55234bd4df1aa8de8156136ff64/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:06.860824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06f74a-f67a-4813-b4e7-cb9beace4412 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:06.861276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06f74a-f67a-4813-b4e7-cb9beace4412 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:06.861389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06f74a-f67a-4813-b4e7-cb9beace4412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:06.888131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-eb06f74a-f67a-4813-b4e7-cb9beace4412 None None] MFA Rules not processed for user `a76dc55234bd4df1aa8de8156136ff64`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:06.889368 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-eb06f74a-f67a-4813-b4e7-cb9beace4412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:06.949246 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:06.959746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1097/2188] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:06.964344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e750a01-a30f-4881-9357-1accd58bf92f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:07.030203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e750a01-a30f-4881-9357-1accd58bf92f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:07.035362 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750167cb-2c24-46a9-a5cb-22d762c68322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:07.036877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750167cb-2c24-46a9-a5cb-22d762c68322 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:07.037490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750167cb-2c24-46a9-a5cb-22d762c68322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:07.037848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750167cb-2c24-46a9-a5cb-22d762c68322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:07.039110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750167cb-2c24-46a9-a5cb-22d762c68322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:07.041625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750167cb-2c24-46a9-a5cb-22d762c68322 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:07.069723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1092/2189] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:06 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:07:07.073890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06f74a-f67a-4813-b4e7-cb9beace4412 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:07.086415 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06f74a-f67a-4813-b4e7-cb9beace4412 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:07.145334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8015eb-58a0-4910-ae57-97507779fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:07.145902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8015eb-58a0-4910-ae57-97507779fe5a admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:07.146179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8015eb-58a0-4910-ae57-97507779fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:07.146449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8015eb-58a0-4910-ae57-97507779fe5a admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:07.147142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8015eb-58a0-4910-ae57-97507779fe5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:07.149112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8015eb-58a0-4910-ae57-97507779fe5a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:07.171855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1098/2190] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:07 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:07.176204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750167cb-2c24-46a9-a5cb-22d762c68322 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:07.188147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750167cb-2c24-46a9-a5cb-22d762c68322 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:07.193021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071ae3a-29e7-46d1-8b4c-d3c4b73ee628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:07.193685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071ae3a-29e7-46d1-8b4c-d3c4b73ee628 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:07.193965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071ae3a-29e7-46d1-8b4c-d3c4b73ee628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:07.194240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071ae3a-29e7-46d1-8b4c-d3c4b73ee628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:07.194990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071ae3a-29e7-46d1-8b4c-d3c4b73ee628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:07.196605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071ae3a-29e7-46d1-8b4c-d3c4b73ee628 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:07.208090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1093/2191] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:07.212206 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8015eb-58a0-4910-ae57-97507779fe5a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:07.222187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8015eb-58a0-4910-ae57-97507779fe5a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:07.228029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813adb8-5bac-4385-9644-7cf38a51358a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:07.228991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813adb8-5bac-4385-9644-7cf38a51358a admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:07.229278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813adb8-5bac-4385-9644-7cf38a51358a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:07.229558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813adb8-5bac-4385-9644-7cf38a51358a admin admin] PATH_INFO: `/v3/projects/a7f08881a64f472e86ff8e9cf9c1883f/users/76ff9334ff50453d821a73954449bcd0/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:07.244775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813adb8-5bac-4385-9644-7cf38a51358a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f08881a64f472e86ff8e9cf9c1883f, user_id=76ff9334ff50453d821a73954449bcd0, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:07.247475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813adb8-5bac-4385-9644-7cf38a51358a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:07.261018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1099/2192] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:07 2022] PUT /identity/v3/projects/a7f08881a64f472e86ff8e9cf9c1883f/users/76ff9334ff50453d821a73954449bcd0/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:07.264980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071ae3a-29e7-46d1-8b4c-d3c4b73ee628 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:07.271936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:07.275915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071ae3a-29e7-46d1-8b4c-d3c4b73ee628 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:07.344691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0d2ba-2cd1-4367-994a-c0d79bd14ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:07.345372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0d2ba-2cd1-4367-994a-c0d79bd14ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:07.345659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0d2ba-2cd1-4367-994a-c0d79bd14ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:07.345939 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0d2ba-2cd1-4367-994a-c0d79bd14ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:07.346671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0d2ba-2cd1-4367-994a-c0d79bd14ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:07.349091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0d2ba-2cd1-4367-994a-c0d79bd14ed7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:07.363552 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1094/2193] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:07.365222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813adb8-5bac-4385-9644-7cf38a51358a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:07.376703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813adb8-5bac-4385-9644-7cf38a51358a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:07.382441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120fad86-21c7-4566-bb21-195e1f81534b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:07.383442 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120fad86-21c7-4566-bb21-195e1f81534b admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:07.383800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120fad86-21c7-4566-bb21-195e1f81534b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:07.384180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120fad86-21c7-4566-bb21-195e1f81534b admin admin] PATH_INFO: `/v3/projects/a7f08881a64f472e86ff8e9cf9c1883f/users/76ff9334ff50453d821a73954449bcd0/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:07.386497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120fad86-21c7-4566-bb21-195e1f81534b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f08881a64f472e86ff8e9cf9c1883f, user_id=76ff9334ff50453d821a73954449bcd0, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:07.389927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120fad86-21c7-4566-bb21-195e1f81534b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:07.403614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1100/2194] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:07 2022] PUT /identity/v3/projects/a7f08881a64f472e86ff8e9cf9c1883f/users/76ff9334ff50453d821a73954449bcd0/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:07.416130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbd580-b3f9-4c9f-aeb8-6fb12d29e4b7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:07.416443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbd580-b3f9-4c9f-aeb8-6fb12d29e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:07.416725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbd580-b3f9-4c9f-aeb8-6fb12d29e4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:07.455287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6bfbd580-b3f9-4c9f-aeb8-6fb12d29e4b7 None None] MFA Rules not processed for user `76ff9334ff50453d821a73954449bcd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:07.456542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6bfbd580-b3f9-4c9f-aeb8-6fb12d29e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:07.526021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1095/2195] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:07.538916 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120fad86-21c7-4566-bb21-195e1f81534b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:07.540691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-120fad86-21c7-4566-bb21-195e1f81534b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:07.598098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120fad86-21c7-4566-bb21-195e1f81534b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:07.601851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d53456-5d8a-469b-a6bc-291eaf1b8f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:07.602495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d53456-5d8a-469b-a6bc-291eaf1b8f3c service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:07.602780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d53456-5d8a-469b-a6bc-291eaf1b8f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:07.603134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d53456-5d8a-469b-a6bc-291eaf1b8f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:07.612160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d53456-5d8a-469b-a6bc-291eaf1b8f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:07.614451 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d53456-5d8a-469b-a6bc-291eaf1b8f3c service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:07.624087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1101/2196] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:07.694148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbd580-b3f9-4c9f-aeb8-6fb12d29e4b7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:07.766262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbd580-b3f9-4c9f-aeb8-6fb12d29e4b7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:07.771578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:07.772501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:07.772790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:07.773073 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] PATH_INFO: `/v3/users/a76dc55234bd4df1aa8de8156136ff64` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:07.774263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76dc55234bd4df1aa8de8156136ff64)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:07.776182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:07.806914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:07.824289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76dc55234bd4df1aa8de8156136ff64'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:07.825005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76dc55234bd4df1aa8de8156136ff64'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:07.832418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76dc55234bd4df1aa8de8156136ff64'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:07.833056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76dc55234bd4df1aa8de8156136ff64'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:07.840028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76dc55234bd4df1aa8de8156136ff64'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:07.857413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1096/2197] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:07 2022] DELETE /identity/v3/users/a76dc55234bd4df1aa8de8156136ff64 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:07.861954 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2114d-8618-4ebe-8a04-759434854456 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:07.862416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2114d-8618-4ebe-8a04-759434854456 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:07.862508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2114d-8618-4ebe-8a04-759434854456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:07.880119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-35d2114d-8618-4ebe-8a04-759434854456 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:07.881343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-35d2114d-8618-4ebe-8a04-759434854456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:07.930306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:07.945693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1102/2198] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:07.955277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:07.956721 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:08.017916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67f8b0-d0a0-4861-92fe-85c40562d0c8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:08.022312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07472d95-413f-4ab5-9e9f-0b2fa577dd16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:08.022957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07472d95-413f-4ab5-9e9f-0b2fa577dd16 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:08.023239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07472d95-413f-4ab5-9e9f-0b2fa577dd16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:08.023510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07472d95-413f-4ab5-9e9f-0b2fa577dd16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:08.032912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07472d95-413f-4ab5-9e9f-0b2fa577dd16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:08.034337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07472d95-413f-4ab5-9e9f-0b2fa577dd16 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:08.044629 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1097/2199] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:08.415426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2114d-8618-4ebe-8a04-759434854456 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:08.418132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-35d2114d-8618-4ebe-8a04-759434854456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:08.449435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2114d-8618-4ebe-8a04-759434854456 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:08.456027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:08.456379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:08.456794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:08.457108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] PATH_INFO: `/v3/projects/a9c252fa9af94e0c90881766e2925b60` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:08.458496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9c252fa9af94e0c90881766e2925b60)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:08.460536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:08.552532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9c252fa9af94e0c90881766e2925b60', 'request_id': 'req-ee011ece-6162-4cac-bfed-fdea7d954386'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:08.560120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9c252fa9af94e0c90881766e2925b60', 'request_id': 'req-ee011ece-6162-4cac-bfed-fdea7d954386'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:08.568841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9c252fa9af94e0c90881766e2925b60', 'request_id': 'req-ee011ece-6162-4cac-bfed-fdea7d954386'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:08.584515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] The token cache is being invalidate because project a9c252fa9af94e0c90881766e2925b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:08.584941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:08.585646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:08.590572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1103/2200] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:08 2022] DELETE /identity/v3/projects/a9c252fa9af94e0c90881766e2925b60 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:08.593447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07472d95-413f-4ab5-9e9f-0b2fa577dd16 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:08.604837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07472d95-413f-4ab5-9e9f-0b2fa577dd16 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:08.683673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:08.685709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:08.686062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:08.687165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] PATH_INFO: `/v3/users/76ff9334ff50453d821a73954449bcd0` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:08.688529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ff9334ff50453d821a73954449bcd0)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:08.690722 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:08.751147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ff9334ff50453d821a73954449bcd0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:08.752754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ff9334ff50453d821a73954449bcd0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:08.756112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:08.767602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ff9334ff50453d821a73954449bcd0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:08.768747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ff9334ff50453d821a73954449bcd0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:08.781970 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b9052e37-ec3b-4ecf-9936-d3d54e08cefc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ff9334ff50453d821a73954449bcd0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:08.806033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1098/2201] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:08 2022] DELETE /identity/v3/users/76ff9334ff50453d821a73954449bcd0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:09.129802 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:09.132363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:09.172895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee011ece-6162-4cac-bfed-fdea7d954386 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:09.259978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:09.260979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:09.261267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:09.261678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] PATH_INFO: `/v3/projects/a7f08881a64f472e86ff8e9cf9c1883f` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:09.262780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7f08881a64f472e86ff8e9cf9c1883f)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:09.265429 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:09.327650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7f08881a64f472e86ff8e9cf9c1883f', 'request_id': 'req-56696283-ca70-4765-9534-ee7b9a9884b0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:09.334718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7f08881a64f472e86ff8e9cf9c1883f', 'request_id': 'req-56696283-ca70-4765-9534-ee7b9a9884b0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:09.343786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7f08881a64f472e86ff8e9cf9c1883f', 'request_id': 'req-56696283-ca70-4765-9534-ee7b9a9884b0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:09.357421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] The token cache is being invalidate because project a7f08881a64f472e86ff8e9cf9c1883f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:09.357760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:09.358489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-56696283-ca70-4765-9534-ee7b9a9884b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:09.360476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1104/2202] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:09 2022] DELETE /identity/v3/projects/a7f08881a64f472e86ff8e9cf9c1883f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:09.368185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca39e6-4d10-477e-88b5-46d99359773a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:09.368185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca39e6-4d10-477e-88b5-46d99359773a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:09.368185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca39e6-4d10-477e-88b5-46d99359773a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:09.392752 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-86ca39e6-4d10-477e-88b5-46d99359773a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:09.393992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-86ca39e6-4d10-477e-88b5-46d99359773a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:09.499624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1099/2203] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:07:09.507711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a4776-637d-42be-98bb-606c7bb94def None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:09.508019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a4776-637d-42be-98bb-606c7bb94def None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:09.508309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a4776-637d-42be-98bb-606c7bb94def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:09.518741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:09.562393 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-dd8a4776-637d-42be-98bb-606c7bb94def None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:09.563793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-dd8a4776-637d-42be-98bb-606c7bb94def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:09.585485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1105/2204] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:09.588603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca39e6-4d10-477e-88b5-46d99359773a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:09.601983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca39e6-4d10-477e-88b5-46d99359773a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:09.624175 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1febbf-ac76-4b13-8598-8651e3e9939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:09.624890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1febbf-ac76-4b13-8598-8651e3e9939b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:09.625173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1febbf-ac76-4b13-8598-8651e3e9939b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:09.625453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1febbf-ac76-4b13-8598-8651e3e9939b admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:09.626181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1febbf-ac76-4b13-8598-8651e3e9939b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:07:09.626554 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1febbf-ac76-4b13-8598-8651e3e9939b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:09.628095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1febbf-ac76-4b13-8598-8651e3e9939b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:09.643608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1100/2205] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:07:09.648767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a4776-637d-42be-98bb-606c7bb94def None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:09.659365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a4776-637d-42be-98bb-606c7bb94def None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:09.664637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ae6c5-e5eb-4338-aa82-2e8757bfb2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:09.668248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ae6c5-e5eb-4338-aa82-2e8757bfb2cb admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:09.668579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ae6c5-e5eb-4338-aa82-2e8757bfb2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:09.671314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ae6c5-e5eb-4338-aa82-2e8757bfb2cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:09.671987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ae6c5-e5eb-4338-aa82-2e8757bfb2cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:09.674095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ae6c5-e5eb-4338-aa82-2e8757bfb2cb admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:09.708433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1106/2206] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:09 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:07:09.711691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1febbf-ac76-4b13-8598-8651e3e9939b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:09.725368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1febbf-ac76-4b13-8598-8651e3e9939b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:09.790943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c9501-53e8-45f2-ba58-98d354837306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:09.791617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9501-53e8-45f2-ba58-98d354837306 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:09.791918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9501-53e8-45f2-ba58-98d354837306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:09.792649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9501-53e8-45f2-ba58-98d354837306 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:09.793077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c9501-53e8-45f2-ba58-98d354837306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:09.795090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c9501-53e8-45f2-ba58-98d354837306 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:09.823936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1101/2207] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:09 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:09.828218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ae6c5-e5eb-4338-aa82-2e8757bfb2cb admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:09.854838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ae6c5-e5eb-4338-aa82-2e8757bfb2cb admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:09.860153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8acdd-3f04-43f1-8072-77c627d3a566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:09.860860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8acdd-3f04-43f1-8072-77c627d3a566 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:09.861144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8acdd-3f04-43f1-8072-77c627d3a566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:09.861417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8acdd-3f04-43f1-8072-77c627d3a566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:09.862195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8acdd-3f04-43f1-8072-77c627d3a566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:09.864725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8acdd-3f04-43f1-8072-77c627d3a566 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:09.882080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1107/2208] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:09.886033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c9501-53e8-45f2-ba58-98d354837306 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:09.897317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c9501-53e8-45f2-ba58-98d354837306 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:09.902109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6d62e-bbc0-4089-b854-cf2b2f9e2590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:09.902770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6d62e-bbc0-4089-b854-cf2b2f9e2590 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:09.903054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6d62e-bbc0-4089-b854-cf2b2f9e2590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:09.903328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6d62e-bbc0-4089-b854-cf2b2f9e2590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:09.904114 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6d62e-bbc0-4089-b854-cf2b2f9e2590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:09.905741 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6d62e-bbc0-4089-b854-cf2b2f9e2590 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:09.928132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1102/2209] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:09.929769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8acdd-3f04-43f1-8072-77c627d3a566 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:09.940474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8acdd-3f04-43f1-8072-77c627d3a566 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:09.955303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d284d-090a-4c2f-8332-7a8ec11d7b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:09.956254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d284d-090a-4c2f-8332-7a8ec11d7b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:09.956840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d284d-090a-4c2f-8332-7a8ec11d7b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:09.957866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d284d-090a-4c2f-8332-7a8ec11d7b93 admin admin] PATH_INFO: `/v3/projects/b5421ddf567f4a2a9f475e2117c23999/users/b3a373acd9844e32bcc80f562e24fd7a/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:09.973701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d284d-090a-4c2f-8332-7a8ec11d7b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5421ddf567f4a2a9f475e2117c23999, user_id=b3a373acd9844e32bcc80f562e24fd7a, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:09.976422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d284d-090a-4c2f-8332-7a8ec11d7b93 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:09.994390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1108/2210] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:09 2022] PUT /identity/v3/projects/b5421ddf567f4a2a9f475e2117c23999/users/b3a373acd9844e32bcc80f562e24fd7a/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:10.000047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e93284-a1b7-44b2-beae-b63bb96d62f2 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:10.000345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e93284-a1b7-44b2-beae-b63bb96d62f2 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:10.000690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e93284-a1b7-44b2-beae-b63bb96d62f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:10.029809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-89e93284-a1b7-44b2-beae-b63bb96d62f2 None None] MFA Rules not processed for user `b3a373acd9844e32bcc80f562e24fd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:10.031081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-89e93284-a1b7-44b2-beae-b63bb96d62f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:10.052259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:10.122210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1103/2211] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:10.126771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d284d-090a-4c2f-8332-7a8ec11d7b93 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:10.196352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d284d-090a-4c2f-8332-7a8ec11d7b93 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:10.201173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc63a3e-adf8-4cf7-a02d-ae221502495d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:10.201845 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc63a3e-adf8-4cf7-a02d-ae221502495d admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:10.202131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc63a3e-adf8-4cf7-a02d-ae221502495d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:10.202410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc63a3e-adf8-4cf7-a02d-ae221502495d admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:10.203112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc63a3e-adf8-4cf7-a02d-ae221502495d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:10.205087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc63a3e-adf8-4cf7-a02d-ae221502495d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:10.237711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1109/2212] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:10 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:10.243113 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e93284-a1b7-44b2-beae-b63bb96d62f2 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:10.256429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e93284-a1b7-44b2-beae-b63bb96d62f2 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:10.378915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ca30b-229a-4ef2-b7f2-79164f34369d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:10.380821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ca30b-229a-4ef2-b7f2-79164f34369d admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:10.381106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ca30b-229a-4ef2-b7f2-79164f34369d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:10.382088 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ca30b-229a-4ef2-b7f2-79164f34369d admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:10.382415 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ca30b-229a-4ef2-b7f2-79164f34369d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:10.384782 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ca30b-229a-4ef2-b7f2-79164f34369d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:10.407898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1104/2213] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:10 2022] POST /identity/v3/users => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:07:10.411927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc63a3e-adf8-4cf7-a02d-ae221502495d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:10.423021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc63a3e-adf8-4cf7-a02d-ae221502495d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:10.428638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa8c6d-0b6d-4943-851d-eb46017568d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:10.429315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa8c6d-0b6d-4943-851d-eb46017568d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:10.429604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa8c6d-0b6d-4943-851d-eb46017568d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:10.429913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa8c6d-0b6d-4943-851d-eb46017568d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:10.430653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa8c6d-0b6d-4943-851d-eb46017568d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:10.432271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa8c6d-0b6d-4943-851d-eb46017568d9 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:10.447532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1110/2214] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:10.448775 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ca30b-229a-4ef2-b7f2-79164f34369d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:10.460015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ca30b-229a-4ef2-b7f2-79164f34369d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:10.464783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26f38a-4394-446f-811b-6d59b89188d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:10.465674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26f38a-4394-446f-811b-6d59b89188d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:10.465958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26f38a-4394-446f-811b-6d59b89188d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:10.466232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26f38a-4394-446f-811b-6d59b89188d0 admin admin] PATH_INFO: `/v3/projects/73aa135ccef34dfc8cab60ae49b7f97c/users/ff76f2defacf4f06ad8af8bcac0e3a54/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:10.481172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26f38a-4394-446f-811b-6d59b89188d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73aa135ccef34dfc8cab60ae49b7f97c, user_id=ff76f2defacf4f06ad8af8bcac0e3a54, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:10.483820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26f38a-4394-446f-811b-6d59b89188d0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:10.496738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1105/2215] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:10 2022] PUT /identity/v3/projects/73aa135ccef34dfc8cab60ae49b7f97c/users/ff76f2defacf4f06ad8af8bcac0e3a54/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:10.500714 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa8c6d-0b6d-4943-851d-eb46017568d9 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:10.511454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa8c6d-0b6d-4943-851d-eb46017568d9 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:10.567447 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a9c4f-9595-433f-80e8-d61a02c56f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:10.568107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9c4f-9595-433f-80e8-d61a02c56f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:10.568390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9c4f-9595-433f-80e8-d61a02c56f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:10.568690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9c4f-9595-433f-80e8-d61a02c56f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:10.569416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a9c4f-9595-433f-80e8-d61a02c56f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:10.571014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a9c4f-9595-433f-80e8-d61a02c56f75 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:10.580691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1111/2216] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:10.584849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26f38a-4394-446f-811b-6d59b89188d0 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:10.595226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26f38a-4394-446f-811b-6d59b89188d0 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:10.600893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef0c0d-99a6-4be7-bfbf-ef48285eaf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:10.601783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef0c0d-99a6-4be7-bfbf-ef48285eaf4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:10.602070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef0c0d-99a6-4be7-bfbf-ef48285eaf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:10.602348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef0c0d-99a6-4be7-bfbf-ef48285eaf4f admin admin] PATH_INFO: `/v3/projects/73aa135ccef34dfc8cab60ae49b7f97c/users/ff76f2defacf4f06ad8af8bcac0e3a54/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:10.604343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef0c0d-99a6-4be7-bfbf-ef48285eaf4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=73aa135ccef34dfc8cab60ae49b7f97c, user_id=ff76f2defacf4f06ad8af8bcac0e3a54, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:10.607062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef0c0d-99a6-4be7-bfbf-ef48285eaf4f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:10.619804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1106/2217] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:10 2022] PUT /identity/v3/projects/73aa135ccef34dfc8cab60ae49b7f97c/users/ff76f2defacf4f06ad8af8bcac0e3a54/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:10.633070 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cc7c0-6abf-43d3-969b-26744c15a5dd None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:10.633560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cc7c0-6abf-43d3-969b-26744c15a5dd None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:10.633694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cc7c0-6abf-43d3-969b-26744c15a5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:10.653596 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:10.664373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-954cc7c0-6abf-43d3-969b-26744c15a5dd None None] MFA Rules not processed for user `ff76f2defacf4f06ad8af8bcac0e3a54`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:10.665812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-954cc7c0-6abf-43d3-969b-26744c15a5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:10.747980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1112/2218] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:10.760162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef0c0d-99a6-4be7-bfbf-ef48285eaf4f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:10.762065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-24ef0c0d-99a6-4be7-bfbf-ef48285eaf4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:10.826891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef0c0d-99a6-4be7-bfbf-ef48285eaf4f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:10.830551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4658c-7bba-4f8f-82db-7b61425c4cdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:10.831292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4658c-7bba-4f8f-82db-7b61425c4cdb service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:10.831582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4658c-7bba-4f8f-82db-7b61425c4cdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:10.832380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4658c-7bba-4f8f-82db-7b61425c4cdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:10.842069 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4658c-7bba-4f8f-82db-7b61425c4cdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:10.843578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4658c-7bba-4f8f-82db-7b61425c4cdb service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:10.854543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1107/2219] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:07:10.956799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cc7c0-6abf-43d3-969b-26744c15a5dd None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:10.966604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cc7c0-6abf-43d3-969b-26744c15a5dd None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:10.970300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02fe0b-cf4d-4676-b84c-2902a5a3e050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:10.970930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02fe0b-cf4d-4676-b84c-2902a5a3e050 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:10.971213 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02fe0b-cf4d-4676-b84c-2902a5a3e050 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:10.971488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02fe0b-cf4d-4676-b84c-2902a5a3e050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:11.035515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02fe0b-cf4d-4676-b84c-2902a5a3e050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:11.037052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02fe0b-cf4d-4676-b84c-2902a5a3e050 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:11.047880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1113/2220] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:12.007412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4658c-7bba-4f8f-82db-7b61425c4cdb service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:12.008893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c3f4658c-7bba-4f8f-82db-7b61425c4cdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:12.044027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:12.092737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4658c-7bba-4f8f-82db-7b61425c4cdb service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:12.097336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:12.098381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:12.098697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:12.098979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] PATH_INFO: `/v3/users/927b174230834bb585671281d6305b3d` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:12.100208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=927b174230834bb585671281d6305b3d)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:12.102186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:12.154543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927b174230834bb585671281d6305b3d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:12.155246 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927b174230834bb585671281d6305b3d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:12.163405 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927b174230834bb585671281d6305b3d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:12.164075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927b174230834bb585671281d6305b3d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:12.172167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927b174230834bb585671281d6305b3d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:12.191891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1108/2221] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:12 2022] DELETE /identity/v3/users/927b174230834bb585671281d6305b3d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:12.197488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3394e-edee-4a2d-b8f5-285d43c5cb52 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:12.197938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3394e-edee-4a2d-b8f5-285d43c5cb52 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:12.198054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3394e-edee-4a2d-b8f5-285d43c5cb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:12.214829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-a2b3394e-edee-4a2d-b8f5-285d43c5cb52 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:12.216076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a2b3394e-edee-4a2d-b8f5-285d43c5cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:12.279449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1114/2222] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:12.290103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:12.291494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:12.351215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbe94b-db88-44e5-91dd-d65fe23c2263 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:12.355426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7267d7e-db7c-43d4-9333-4853b5179d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:12.356200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7267d7e-db7c-43d4-9333-4853b5179d5c service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:12.356520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7267d7e-db7c-43d4-9333-4853b5179d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:12.356806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7267d7e-db7c-43d4-9333-4853b5179d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:12.367582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7267d7e-db7c-43d4-9333-4853b5179d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:12.369080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7267d7e-db7c-43d4-9333-4853b5179d5c service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:12.378965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1109/2223] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:12.798478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3394e-edee-4a2d-b8f5-285d43c5cb52 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:12.800240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a2b3394e-edee-4a2d-b8f5-285d43c5cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:12.812385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3394e-edee-4a2d-b8f5-285d43c5cb52 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:12.828649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:12.829536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:12.829826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:12.830165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] PATH_INFO: `/v3/projects/d641a85e873144a981b4c1fc60c17db3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:12.831924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d641a85e873144a981b4c1fc60c17db3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:12.834044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:12.874962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:12.892313 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd641a85e873144a981b4c1fc60c17db3', 'request_id': 'req-be315d91-1d8d-424b-87a3-6bd223b5855e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:12.899987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd641a85e873144a981b4c1fc60c17db3', 'request_id': 'req-be315d91-1d8d-424b-87a3-6bd223b5855e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:12.906841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd641a85e873144a981b4c1fc60c17db3', 'request_id': 'req-be315d91-1d8d-424b-87a3-6bd223b5855e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:12.928498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] The token cache is being invalidate because project d641a85e873144a981b4c1fc60c17db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:12.928837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:12.929466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:12.931171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1115/2224] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:12 2022] DELETE /identity/v3/projects/d641a85e873144a981b4c1fc60c17db3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:12.935242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7267d7e-db7c-43d4-9333-4853b5179d5c service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:12.946232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7267d7e-db7c-43d4-9333-4853b5179d5c service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:12.985122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:13.010881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:13.011745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:13.012032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:13.012310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] PATH_INFO: `/v3/users/2794940b4d8542c888c30ece8c471aee` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:13.013548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2794940b4d8542c888c30ece8c471aee)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:13.015505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:13.066549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2794940b4d8542c888c30ece8c471aee'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.067204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2794940b4d8542c888c30ece8c471aee'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.073989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2794940b4d8542c888c30ece8c471aee'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.074643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2794940b4d8542c888c30ece8c471aee'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.082670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-689d94b2-bab1-4075-bb9f-c1acebf3762d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2794940b4d8542c888c30ece8c471aee'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.101714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1110/2225] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:12 2022] DELETE /identity/v3/users/2794940b4d8542c888c30ece8c471aee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:13.460468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:13.461939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:13.487813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be315d91-1d8d-424b-87a3-6bd223b5855e admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:13.551722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:13.552622 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:13.552908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:13.553181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] PATH_INFO: `/v3/projects/ee4fafbcccc24d9aad00975656b00e04` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:13.554356 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee4fafbcccc24d9aad00975656b00e04)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:13.556293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:13.603293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee4fafbcccc24d9aad00975656b00e04', 'request_id': 'req-496cf437-3f38-414e-a790-4e0aa60cce9a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.608743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee4fafbcccc24d9aad00975656b00e04', 'request_id': 'req-496cf437-3f38-414e-a790-4e0aa60cce9a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.615185 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee4fafbcccc24d9aad00975656b00e04', 'request_id': 'req-496cf437-3f38-414e-a790-4e0aa60cce9a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.627358 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] The token cache is being invalidate because project ee4fafbcccc24d9aad00975656b00e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:13.627650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.628284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-496cf437-3f38-414e-a790-4e0aa60cce9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.630382 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1116/2226] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:13 2022] DELETE /identity/v3/projects/ee4fafbcccc24d9aad00975656b00e04 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:13.635293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb83a3e-eb54-4697-a705-2cc6fc1ac29e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:13.635585 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb83a3e-eb54-4697-a705-2cc6fc1ac29e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:13.635865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb83a3e-eb54-4697-a705-2cc6fc1ac29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:13.654295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3fb83a3e-eb54-4697-a705-2cc6fc1ac29e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:13.655442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3fb83a3e-eb54-4697-a705-2cc6fc1ac29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:13.718591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1111/2227] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:13.730378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c2d5-6634-416f-85c1-aecb0cf30877 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:13.730926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c2d5-6634-416f-85c1-aecb0cf30877 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:13.731266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c2d5-6634-416f-85c1-aecb0cf30877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:13.733581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb83a3e-eb54-4697-a705-2cc6fc1ac29e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:13.735066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3fb83a3e-eb54-4697-a705-2cc6fc1ac29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:13.749982 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb83a3e-eb54-4697-a705-2cc6fc1ac29e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:13.751482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-9b15c2d5-6634-416f-85c1-aecb0cf30877 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:13.752863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-9b15c2d5-6634-416f-85c1-aecb0cf30877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:13.757090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:13.758203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:13.758325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:13.758539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] PATH_INFO: `/v3/users/b3a373acd9844e32bcc80f562e24fd7a` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:13.759733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a373acd9844e32bcc80f562e24fd7a)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:13.761685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:13.762341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1117/2228] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:13.766243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c2d5-6634-416f-85c1-aecb0cf30877 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:13.776977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c2d5-6634-416f-85c1-aecb0cf30877 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:13.783894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500e13c-8b88-4e75-a0ac-6d852a8fdc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:13.785118 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500e13c-8b88-4e75-a0ac-6d852a8fdc10 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:13.785656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500e13c-8b88-4e75-a0ac-6d852a8fdc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:13.786032 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500e13c-8b88-4e75-a0ac-6d852a8fdc10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:13.786879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500e13c-8b88-4e75-a0ac-6d852a8fdc10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:07:13.787293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500e13c-8b88-4e75-a0ac-6d852a8fdc10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:13.789006 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500e13c-8b88-4e75-a0ac-6d852a8fdc10 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:13.804354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1118/2229] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 02:07:13.808613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500e13c-8b88-4e75-a0ac-6d852a8fdc10 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:13.822495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500e13c-8b88-4e75-a0ac-6d852a8fdc10 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:13.827219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a373acd9844e32bcc80f562e24fd7a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.828013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a373acd9844e32bcc80f562e24fd7a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.828582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe63be-5dce-4299-b533-be4d4ae659ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:13.829686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe63be-5dce-4299-b533-be4d4ae659ba admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:13.830181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe63be-5dce-4299-b533-be4d4ae659ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:13.830735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe63be-5dce-4299-b533-be4d4ae659ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:13.831991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe63be-5dce-4299-b533-be4d4ae659ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:13.834273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe63be-5dce-4299-b533-be4d4ae659ba admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:13.835333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:13.838896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a373acd9844e32bcc80f562e24fd7a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.841029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a373acd9844e32bcc80f562e24fd7a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.852562 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b7e6a97a-156c-4aa9-bd64-3aabcfc9e991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a373acd9844e32bcc80f562e24fd7a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:13.885185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1112/2230] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:13 2022] DELETE /identity/v3/users/b3a373acd9844e32bcc80f562e24fd7a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:13.887429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58625ee1-eb5e-443d-a8c9-275229bda96a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:13.887723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58625ee1-eb5e-443d-a8c9-275229bda96a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:13.887997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58625ee1-eb5e-443d-a8c9-275229bda96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:13.891460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1119/2231] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:13 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:13.892491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe63be-5dce-4299-b533-be4d4ae659ba admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:13.894765 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5bbe63be-5dce-4299-b533-be4d4ae659ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:13.910554 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:13.947598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-58625ee1-eb5e-443d-a8c9-275229bda96a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:13.948999 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-58625ee1-eb5e-443d-a8c9-275229bda96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:14.004488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe63be-5dce-4299-b533-be4d4ae659ba admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.009550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c0358-6304-4822-be82-619b4376f7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.010233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c0358-6304-4822-be82-619b4376f7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.010521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c0358-6304-4822-be82-619b4376f7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.010800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c0358-6304-4822-be82-619b4376f7d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.013494 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c0358-6304-4822-be82-619b4376f7d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.016066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c0358-6304-4822-be82-619b4376f7d0 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.044053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1120/2232] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:13 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:14.047858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c0358-6304-4822-be82-619b4376f7d0 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.079175 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1113/2233] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:07:14.096078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58625ee1-eb5e-443d-a8c9-275229bda96a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.098252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-58625ee1-eb5e-443d-a8c9-275229bda96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:14.107249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c0358-6304-4822-be82-619b4376f7d0 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.112171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118ed91-7e8e-450e-9431-ad7c9b88dc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.113310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118ed91-7e8e-450e-9431-ad7c9b88dc30 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.113310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118ed91-7e8e-450e-9431-ad7c9b88dc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.113669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118ed91-7e8e-450e-9431-ad7c9b88dc30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.114592 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118ed91-7e8e-450e-9431-ad7c9b88dc30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.116378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118ed91-7e8e-450e-9431-ad7c9b88dc30 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.127271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1121/2234] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:14.131687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118ed91-7e8e-450e-9431-ad7c9b88dc30 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.141860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118ed91-7e8e-450e-9431-ad7c9b88dc30 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.147360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edf91d-e354-4da8-be5b-b13f406cf845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.148040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edf91d-e354-4da8-be5b-b13f406cf845 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.148315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edf91d-e354-4da8-be5b-b13f406cf845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.148621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edf91d-e354-4da8-be5b-b13f406cf845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.149343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62edf91d-e354-4da8-be5b-b13f406cf845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.150935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62edf91d-e354-4da8-be5b-b13f406cf845 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.170086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1122/2235] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:14.173784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edf91d-e354-4da8-be5b-b13f406cf845 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.184219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edf91d-e354-4da8-be5b-b13f406cf845 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.189210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4856374-bba6-447b-9892-4d360fd0a117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.190091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4856374-bba6-447b-9892-4d360fd0a117 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.190372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4856374-bba6-447b-9892-4d360fd0a117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.190798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4856374-bba6-447b-9892-4d360fd0a117 admin admin] PATH_INFO: `/v3/projects/745b66b72c1d49aca57c58f83111f348/users/554816a509b249aaa920cdce385466b6/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.199792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:14.205086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58625ee1-eb5e-443d-a8c9-275229bda96a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.211481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4856374-bba6-447b-9892-4d360fd0a117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=745b66b72c1d49aca57c58f83111f348, user_id=554816a509b249aaa920cdce385466b6, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.214342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4856374-bba6-447b-9892-4d360fd0a117 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.218190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea20e97-ab03-4c12-97d4-afe148845628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.218852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea20e97-ab03-4c12-97d4-afe148845628 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.219131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea20e97-ab03-4c12-97d4-afe148845628 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.219400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea20e97-ab03-4c12-97d4-afe148845628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.232629 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea20e97-ab03-4c12-97d4-afe148845628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.234082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea20e97-ab03-4c12-97d4-afe148845628 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.239848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1123/2236] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:14 2022] PUT /identity/v3/projects/745b66b72c1d49aca57c58f83111f348/users/554816a509b249aaa920cdce385466b6/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:14.247201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1114/2237] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:07:14.255029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62de9d-9b16-4988-bae0-ae04e9f8291f None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.255450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62de9d-9b16-4988-bae0-ae04e9f8291f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.255828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62de9d-9b16-4988-bae0-ae04e9f8291f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.293960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-4d62de9d-9b16-4988-bae0-ae04e9f8291f None None] MFA Rules not processed for user `554816a509b249aaa920cdce385466b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:14.295254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4d62de9d-9b16-4988-bae0-ae04e9f8291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:14.387505 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:14.399734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1124/2238] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:14.401441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea20e97-ab03-4c12-97d4-afe148845628 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.415373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea20e97-ab03-4c12-97d4-afe148845628 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.507386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e37367-b986-470a-946a-ebb6835beb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.508193 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e37367-b986-470a-946a-ebb6835beb42 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.508594 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e37367-b986-470a-946a-ebb6835beb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.509538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e37367-b986-470a-946a-ebb6835beb42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.510326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e37367-b986-470a-946a-ebb6835beb42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.512505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e37367-b986-470a-946a-ebb6835beb42 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.549323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1115/2239] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:14 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:14.550068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62de9d-9b16-4988-bae0-ae04e9f8291f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.566678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62de9d-9b16-4988-bae0-ae04e9f8291f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.646981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40076c-d66c-46eb-baf8-d0e35227378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.647725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40076c-d66c-46eb-baf8-d0e35227378c admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.648085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40076c-d66c-46eb-baf8-d0e35227378c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.648469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40076c-d66c-46eb-baf8-d0e35227378c admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.649244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40076c-d66c-46eb-baf8-d0e35227378c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.651287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40076c-d66c-46eb-baf8-d0e35227378c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.685635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1125/2240] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:14 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:07:14.693321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e37367-b986-470a-946a-ebb6835beb42 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.700067 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40076c-d66c-46eb-baf8-d0e35227378c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.703012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3c40076c-d66c-46eb-baf8-d0e35227378c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:14.705831 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e37367-b986-470a-946a-ebb6835beb42 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.712269 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c4be0-0bdd-459f-a4f5-44232c1382d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.713369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4be0-0bdd-459f-a4f5-44232c1382d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.713849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4be0-0bdd-459f-a4f5-44232c1382d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.714229 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4be0-0bdd-459f-a4f5-44232c1382d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.715068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c4be0-0bdd-459f-a4f5-44232c1382d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.718076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c4be0-0bdd-459f-a4f5-44232c1382d2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.725519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40076c-d66c-46eb-baf8-d0e35227378c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.736253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.736486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1116/2241] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:14.736593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.736593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.736593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] PATH_INFO: `/v3/projects/b5421ddf567f4a2a9f475e2117c23999` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.737219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5421ddf567f4a2a9f475e2117c23999)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.739254 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c4be0-0bdd-459f-a4f5-44232c1382d2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.740024 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.753088 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c4be0-0bdd-459f-a4f5-44232c1382d2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.758036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833df6c-65e3-479d-80c1-6eaca40e0b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.758710 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833df6c-65e3-479d-80c1-6eaca40e0b65 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.758992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833df6c-65e3-479d-80c1-6eaca40e0b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.759266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833df6c-65e3-479d-80c1-6eaca40e0b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.759998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833df6c-65e3-479d-80c1-6eaca40e0b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.761706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833df6c-65e3-479d-80c1-6eaca40e0b65 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.773510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1117/2242] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:14.778833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833df6c-65e3-479d-80c1-6eaca40e0b65 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.789981 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833df6c-65e3-479d-80c1-6eaca40e0b65 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.797931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5421ddf567f4a2a9f475e2117c23999', 'request_id': 'req-dc762edc-103c-4ca1-a6fc-a625b67ba654'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:14.808665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5421ddf567f4a2a9f475e2117c23999', 'request_id': 'req-dc762edc-103c-4ca1-a6fc-a625b67ba654'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:14.817801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5421ddf567f4a2a9f475e2117c23999', 'request_id': 'req-dc762edc-103c-4ca1-a6fc-a625b67ba654'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:14.839394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] The token cache is being invalidate because project b5421ddf567f4a2a9f475e2117c23999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:14.840086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:14.841011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:14.843291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1126/2243] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:14 2022] DELETE /identity/v3/projects/b5421ddf567f4a2a9f475e2117c23999 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:14.846868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:14.848908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:14.857713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1279a-c51d-46b5-9c2f-07f59503e3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:14.858686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1279a-c51d-46b5-9c2f-07f59503e3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.858913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1279a-c51d-46b5-9c2f-07f59503e3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.859138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1279a-c51d-46b5-9c2f-07f59503e3ea admin admin] PATH_INFO: `/v3/projects/a850ace674e341db952db6f3a541779e/users/90f6c868c26a450f8c54d57600fbcb1d/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.878986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1279a-c51d-46b5-9c2f-07f59503e3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a850ace674e341db952db6f3a541779e, user_id=90f6c868c26a450f8c54d57600fbcb1d, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:14.882772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1279a-c51d-46b5-9c2f-07f59503e3ea admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:14.919674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1118/2244] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:14 2022] PUT /identity/v3/projects/a850ace674e341db952db6f3a541779e/users/90f6c868c26a450f8c54d57600fbcb1d/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:14.922234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66915d-7678-4b01-96a0-57dcce6cc3b0 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:14.924136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66915d-7678-4b01-96a0-57dcce6cc3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:14.924420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66915d-7678-4b01-96a0-57dcce6cc3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:14.977502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3b66915d-7678-4b01-96a0-57dcce6cc3b0 None None] MFA Rules not processed for user `90f6c868c26a450f8c54d57600fbcb1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:14.979296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3b66915d-7678-4b01-96a0-57dcce6cc3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:14.985808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc762edc-103c-4ca1-a6fc-a625b67ba654 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:14.999983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:15.014315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:15.060912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:15.061789 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.062071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.062350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] PATH_INFO: `/v3/users/ff76f2defacf4f06ad8af8bcac0e3a54` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.063683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff76f2defacf4f06ad8af8bcac0e3a54)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:15.065839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:15.069599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1119/2245] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:15.070621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66915d-7678-4b01-96a0-57dcce6cc3b0 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:15.071940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3b66915d-7678-4b01-96a0-57dcce6cc3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:15.086806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66915d-7678-4b01-96a0-57dcce6cc3b0 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:15.093992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a9f7b-65de-4187-8870-425ccfb9e951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:15.094762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a9f7b-65de-4187-8870-425ccfb9e951 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.095049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a9f7b-65de-4187-8870-425ccfb9e951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.095323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a9f7b-65de-4187-8870-425ccfb9e951 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.096011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a9f7b-65de-4187-8870-425ccfb9e951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:15.098698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a9f7b-65de-4187-8870-425ccfb9e951 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:15.125025 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff76f2defacf4f06ad8af8bcac0e3a54'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:15.125903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff76f2defacf4f06ad8af8bcac0e3a54'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:15.135870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1120/2246] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:15 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:15.140837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a9f7b-65de-4187-8870-425ccfb9e951 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:15.142549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff76f2defacf4f06ad8af8bcac0e3a54'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:15.143703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff76f2defacf4f06ad8af8bcac0e3a54'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:15.157316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff76f2defacf4f06ad8af8bcac0e3a54'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:15.174469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1127/2247] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:14 2022] DELETE /identity/v3/users/ff76f2defacf4f06ad8af8bcac0e3a54 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:15.225984 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a9f7b-65de-4187-8870-425ccfb9e951 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:15.321432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5b41c-f1e2-443b-80d6-5263586fea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:15.322093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5b41c-f1e2-443b-80d6-5263586fea59 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.322380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5b41c-f1e2-443b-80d6-5263586fea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.322761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5b41c-f1e2-443b-80d6-5263586fea59 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.323606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5b41c-f1e2-443b-80d6-5263586fea59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:15.325569 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5b41c-f1e2-443b-80d6-5263586fea59 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:15.352308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1121/2248] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:15 2022] POST /identity/v3/users => generated 295 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:07:15.355955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:15.357436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:15.371442 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7651c-5d92-4dcf-b359-fe3a4015e855 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:15.376546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafac9b-6de0-4722-ba3c-863bcb57e15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:15.378139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafac9b-6de0-4722-ba3c-863bcb57e15a admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.378139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafac9b-6de0-4722-ba3c-863bcb57e15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.378139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafac9b-6de0-4722-ba3c-863bcb57e15a admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.378795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafac9b-6de0-4722-ba3c-863bcb57e15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:15.380426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafac9b-6de0-4722-ba3c-863bcb57e15a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:15.395869 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1128/2249] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:15.397393 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5b41c-f1e2-443b-80d6-5263586fea59 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:15.410910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5b41c-f1e2-443b-80d6-5263586fea59 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:15.416346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acc074-b762-4400-b429-4eef265cbf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:15.417407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acc074-b762-4400-b429-4eef265cbf9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.417804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acc074-b762-4400-b429-4eef265cbf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.418148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acc074-b762-4400-b429-4eef265cbf9f admin admin] PATH_INFO: `/v3/projects/8e37344d771444b2ac119e4a03e50bb6/users/c29563c76e644976a82c4035cf0db547/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.440473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acc074-b762-4400-b429-4eef265cbf9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e37344d771444b2ac119e4a03e50bb6, user_id=c29563c76e644976a82c4035cf0db547, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:15.443325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acc074-b762-4400-b429-4eef265cbf9f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:15.463087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1122/2250] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:15 2022] PUT /identity/v3/projects/8e37344d771444b2ac119e4a03e50bb6/users/c29563c76e644976a82c4035cf0db547/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:15.466983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafac9b-6de0-4722-ba3c-863bcb57e15a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:15.482553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafac9b-6de0-4722-ba3c-863bcb57e15a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:15.551408 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acc074-b762-4400-b429-4eef265cbf9f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:15.553600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-95acc074-b762-4400-b429-4eef265cbf9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:15.593931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:15.605579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5aac1-57b5-459e-8f39-1467122181b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:15.606282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5aac1-57b5-459e-8f39-1467122181b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.606572 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5aac1-57b5-459e-8f39-1467122181b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.609865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5aac1-57b5-459e-8f39-1467122181b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.610623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5aac1-57b5-459e-8f39-1467122181b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:15.612248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5aac1-57b5-459e-8f39-1467122181b1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:15.629544 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1129/2251] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:15.633991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5aac1-57b5-459e-8f39-1467122181b1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:15.647297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5aac1-57b5-459e-8f39-1467122181b1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:15.654192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75e15f-348f-449c-b1d0-2f47d215ff80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:15.655253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75e15f-348f-449c-b1d0-2f47d215ff80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.656252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75e15f-348f-449c-b1d0-2f47d215ff80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.656588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75e15f-348f-449c-b1d0-2f47d215ff80 admin admin] PATH_INFO: `/v3/projects/8e37344d771444b2ac119e4a03e50bb6/users/c29563c76e644976a82c4035cf0db547/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.658715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75e15f-348f-449c-b1d0-2f47d215ff80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e37344d771444b2ac119e4a03e50bb6, user_id=c29563c76e644976a82c4035cf0db547, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:15.662638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75e15f-348f-449c-b1d0-2f47d215ff80 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:15.667631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acc074-b762-4400-b429-4eef265cbf9f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:15.672789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:15.673643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.673929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.674341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] PATH_INFO: `/v3/projects/73aa135ccef34dfc8cab60ae49b7f97c` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.675676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73aa135ccef34dfc8cab60ae49b7f97c)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:15.677811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:15.680451 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1130/2252] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:15 2022] PUT /identity/v3/projects/8e37344d771444b2ac119e4a03e50bb6/users/c29563c76e644976a82c4035cf0db547/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:15.692290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9fab0-3a15-482f-a07f-0fadef6d82c8 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.694035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9fab0-3a15-482f-a07f-0fadef6d82c8 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.694335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9fab0-3a15-482f-a07f-0fadef6d82c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.727986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:15.747171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7ee9fab0-3a15-482f-a07f-0fadef6d82c8 None None] MFA Rules not processed for user `c29563c76e644976a82c4035cf0db547`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:15.750423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7ee9fab0-3a15-482f-a07f-0fadef6d82c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:15.758941 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73aa135ccef34dfc8cab60ae49b7f97c', 'request_id': 'req-67f1033a-826f-4ad6-bf54-788721b86642'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:15.767379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73aa135ccef34dfc8cab60ae49b7f97c', 'request_id': 'req-67f1033a-826f-4ad6-bf54-788721b86642'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:15.779746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73aa135ccef34dfc8cab60ae49b7f97c', 'request_id': 'req-67f1033a-826f-4ad6-bf54-788721b86642'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:15.802705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] The token cache is being invalidate because project 73aa135ccef34dfc8cab60ae49b7f97c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:15.803059 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:15.803691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-67f1033a-826f-4ad6-bf54-788721b86642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:15.808964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1123/2253] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:15 2022] DELETE /identity/v3/projects/73aa135ccef34dfc8cab60ae49b7f97c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:15.810452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a05ba-35aa-43a1-9807-969748305b5b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.810745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a05ba-35aa-43a1-9807-969748305b5b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.811025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a05ba-35aa-43a1-9807-969748305b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.863386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1131/2254] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:07:15.871201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9fab0-3a15-482f-a07f-0fadef6d82c8 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:15.873204 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7ee9fab0-3a15-482f-a07f-0fadef6d82c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:15.886485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ca9a05ba-35aa-43a1-9807-969748305b5b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:15.888364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ca9a05ba-35aa-43a1-9807-969748305b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:15.952247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9fab0-3a15-482f-a07f-0fadef6d82c8 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:15.961329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6ee2a-f042-4fd9-aa7c-4e064d07f9bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:15.962377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6ee2a-f042-4fd9-aa7c-4e064d07f9bf service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:15.962842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6ee2a-f042-4fd9-aa7c-4e064d07f9bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:15.963828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6ee2a-f042-4fd9-aa7c-4e064d07f9bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:15.966175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-51a6ee2a-f042-4fd9-aa7c-4e064d07f9bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:16.000804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1124/2255] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:16.009154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63cf8b-5ef7-4084-9305-1a40dc061d4b None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.009446 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63cf8b-5ef7-4084-9305-1a40dc061d4b None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.009726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63cf8b-5ef7-4084-9305-1a40dc061d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.055450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4f63cf8b-5ef7-4084-9305-1a40dc061d4b None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:16.057504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4f63cf8b-5ef7-4084-9305-1a40dc061d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:16.070290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1125/2256] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:16.071643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63cf8b-5ef7-4084-9305-1a40dc061d4b None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:16.071762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6ee2a-f042-4fd9-aa7c-4e064d07f9bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.073379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6ee2a-f042-4fd9-aa7c-4e064d07f9bf service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:16.085015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63cf8b-5ef7-4084-9305-1a40dc061d4b None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:16.085656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1132/2257] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:07:16.093167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb19549-3c8d-412b-9a88-b2296d35f21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:16.093713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb19549-3c8d-412b-9a88-b2296d35f21c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.094002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb19549-3c8d-412b-9a88-b2296d35f21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.094648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb19549-3c8d-412b-9a88-b2296d35f21c admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.095429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb19549-3c8d-412b-9a88-b2296d35f21c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:07:16.095733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb19549-3c8d-412b-9a88-b2296d35f21c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.098028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb19549-3c8d-412b-9a88-b2296d35f21c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:16.123832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1126/2258] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 02:07:16.128519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6ee2a-f042-4fd9-aa7c-4e064d07f9bf service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:16.140437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6ee2a-f042-4fd9-aa7c-4e064d07f9bf service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:16.146144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24a46a-13e6-4f8c-87b1-cb9282583822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:16.146826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24a46a-13e6-4f8c-87b1-cb9282583822 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.147108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24a46a-13e6-4f8c-87b1-cb9282583822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.147387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24a46a-13e6-4f8c-87b1-cb9282583822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.148076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24a46a-13e6-4f8c-87b1-cb9282583822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.150054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24a46a-13e6-4f8c-87b1-cb9282583822 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:16.185982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1133/2259] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:16 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:16.188704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb19549-3c8d-412b-9a88-b2296d35f21c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:16.198830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:16.207169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb19549-3c8d-412b-9a88-b2296d35f21c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:16.304536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1dab4-bd61-4ad9-91d2-64f4640f3280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:16.305234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1dab4-bd61-4ad9-91d2-64f4640f3280 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.305517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1dab4-bd61-4ad9-91d2-64f4640f3280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.305797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1dab4-bd61-4ad9-91d2-64f4640f3280 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.306578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1dab4-bd61-4ad9-91d2-64f4640f3280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.308681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1dab4-bd61-4ad9-91d2-64f4640f3280 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:16.344667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1127/2260] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:16 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:16.345125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24a46a-13e6-4f8c-87b1-cb9282583822 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:16.360642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24a46a-13e6-4f8c-87b1-cb9282583822 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:16.374708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68066a9a-339e-44b0-a94b-9f39c7f6ec92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:16.375485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68066a9a-339e-44b0-a94b-9f39c7f6ec92 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.377052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68066a9a-339e-44b0-a94b-9f39c7f6ec92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.377052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68066a9a-339e-44b0-a94b-9f39c7f6ec92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.377552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68066a9a-339e-44b0-a94b-9f39c7f6ec92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.379640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68066a9a-339e-44b0-a94b-9f39c7f6ec92 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:16.401923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1dab4-bd61-4ad9-91d2-64f4640f3280 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:16.410748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1134/2261] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:07:16.414476 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1dab4-bd61-4ad9-91d2-64f4640f3280 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:16.419538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d550e6-876d-48cc-9114-1d0618a49464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:16.420220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d550e6-876d-48cc-9114-1d0618a49464 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.420552 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d550e6-876d-48cc-9114-1d0618a49464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.420833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d550e6-876d-48cc-9114-1d0618a49464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.422172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d550e6-876d-48cc-9114-1d0618a49464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.423991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d550e6-876d-48cc-9114-1d0618a49464 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:16.442161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1128/2262] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:16.443157 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68066a9a-339e-44b0-a94b-9f39c7f6ec92 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:16.459364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68066a9a-339e-44b0-a94b-9f39c7f6ec92 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:16.480247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b78f01-96e9-450e-9fe2-258dfe98bf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:16.481242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b78f01-96e9-450e-9fe2-258dfe98bf4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.481530 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b78f01-96e9-450e-9fe2-258dfe98bf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.481810 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b78f01-96e9-450e-9fe2-258dfe98bf4e admin admin] PATH_INFO: `/v3/projects/b1bd78079b2a4d49b2c3533111621164/users/b979a489e0f94b35b99c7e874eeece46/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.506718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:16.516038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b78f01-96e9-450e-9fe2-258dfe98bf4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1bd78079b2a4d49b2c3533111621164, user_id=b979a489e0f94b35b99c7e874eeece46, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.520256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b78f01-96e9-450e-9fe2-258dfe98bf4e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:16.572102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1135/2263] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:16 2022] PUT /identity/v3/projects/b1bd78079b2a4d49b2c3533111621164/users/b979a489e0f94b35b99c7e874eeece46/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:16.573995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76cdf3-beb7-4d11-8846-2e54be3f021f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.574464 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76cdf3-beb7-4d11-8846-2e54be3f021f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.574901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76cdf3-beb7-4d11-8846-2e54be3f021f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.617498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0e76cdf3-beb7-4d11-8846-2e54be3f021f None None] MFA Rules not processed for user `b979a489e0f94b35b99c7e874eeece46`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:16.619623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0e76cdf3-beb7-4d11-8846-2e54be3f021f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:16.711345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1129/2264] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:16.712690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b78f01-96e9-450e-9fe2-258dfe98bf4e admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:16.783304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b78f01-96e9-450e-9fe2-258dfe98bf4e admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:16.788693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501db851-64dc-4e0e-a60a-78b054f027a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:16.789367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db851-64dc-4e0e-a60a-78b054f027a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.789656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db851-64dc-4e0e-a60a-78b054f027a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.789963 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db851-64dc-4e0e-a60a-78b054f027a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.790661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501db851-64dc-4e0e-a60a-78b054f027a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.792611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501db851-64dc-4e0e-a60a-78b054f027a8 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:16.819944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1136/2265] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:16 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:16.824238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76cdf3-beb7-4d11-8846-2e54be3f021f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:16.835378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76cdf3-beb7-4d11-8846-2e54be3f021f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:16.891192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada3226-f9d8-44e0-a216-37928d1282fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:16.891857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3226-f9d8-44e0-a216-37928d1282fb admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.892143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3226-f9d8-44e0-a216-37928d1282fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.892457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3226-f9d8-44e0-a216-37928d1282fb admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.893137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada3226-f9d8-44e0-a216-37928d1282fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.895413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada3226-f9d8-44e0-a216-37928d1282fb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:16.915934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1130/2266] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:16 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:16.919818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501db851-64dc-4e0e-a60a-78b054f027a8 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:16.931540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501db851-64dc-4e0e-a60a-78b054f027a8 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:16.936217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45063c98-3997-43c2-9e9b-519563fb6750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:16.936917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063c98-3997-43c2-9e9b-519563fb6750 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.937203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063c98-3997-43c2-9e9b-519563fb6750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.937481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063c98-3997-43c2-9e9b-519563fb6750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.938240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45063c98-3997-43c2-9e9b-519563fb6750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.939848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45063c98-3997-43c2-9e9b-519563fb6750 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:16.950848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1137/2267] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:16.954979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada3226-f9d8-44e0-a216-37928d1282fb admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:16.960190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:16.966331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada3226-f9d8-44e0-a216-37928d1282fb admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:16.971375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16d7e2-65b1-4775-af43-6e384ec868a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:16.972468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16d7e2-65b1-4775-af43-6e384ec868a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:16.972784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16d7e2-65b1-4775-af43-6e384ec868a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:16.973060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16d7e2-65b1-4775-af43-6e384ec868a1 admin admin] PATH_INFO: `/v3/projects/2e5d2ab9db994fda8b0455cf5b486803/users/6421931f70604b6a89bc868ef771bf2a/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:16.987392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16d7e2-65b1-4775-af43-6e384ec868a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e5d2ab9db994fda8b0455cf5b486803, user_id=6421931f70604b6a89bc868ef771bf2a, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:16.990691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16d7e2-65b1-4775-af43-6e384ec868a1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:17.002643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1131/2268] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:16 2022] PUT /identity/v3/projects/2e5d2ab9db994fda8b0455cf5b486803/users/6421931f70604b6a89bc868ef771bf2a/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:17.006753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45063c98-3997-43c2-9e9b-519563fb6750 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:17.016684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45063c98-3997-43c2-9e9b-519563fb6750 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:17.087824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912e7e9-4398-48c1-a92a-511084e049c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:17.088616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912e7e9-4398-48c1-a92a-511084e049c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:17.088971 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912e7e9-4398-48c1-a92a-511084e049c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:17.089339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912e7e9-4398-48c1-a92a-511084e049c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:17.090200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912e7e9-4398-48c1-a92a-511084e049c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:17.091939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912e7e9-4398-48c1-a92a-511084e049c5 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:17.102323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1138/2269] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:17.106446 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16d7e2-65b1-4775-af43-6e384ec868a1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:17.117699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16d7e2-65b1-4775-af43-6e384ec868a1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:17.122429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b2de6-1dfd-4726-aa1f-d99ca5fe31e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:17.123323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b2de6-1dfd-4726-aa1f-d99ca5fe31e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:17.123609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b2de6-1dfd-4726-aa1f-d99ca5fe31e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:17.123890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b2de6-1dfd-4726-aa1f-d99ca5fe31e3 admin admin] PATH_INFO: `/v3/projects/2e5d2ab9db994fda8b0455cf5b486803/users/6421931f70604b6a89bc868ef771bf2a/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:17.126170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b2de6-1dfd-4726-aa1f-d99ca5fe31e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e5d2ab9db994fda8b0455cf5b486803, user_id=6421931f70604b6a89bc868ef771bf2a, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:17.129609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b2de6-1dfd-4726-aa1f-d99ca5fe31e3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:17.140634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1132/2270] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:17 2022] PUT /identity/v3/projects/2e5d2ab9db994fda8b0455cf5b486803/users/6421931f70604b6a89bc868ef771bf2a/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:17.210337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f84f9-44f0-4c84-9982-ff84c2538382 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:17.210631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f84f9-44f0-4c84-9982-ff84c2538382 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:17.210911 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f84f9-44f0-4c84-9982-ff84c2538382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:17.238017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-319f84f9-44f0-4c84-9982-ff84c2538382 None None] MFA Rules not processed for user `6421931f70604b6a89bc868ef771bf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:17.239370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-319f84f9-44f0-4c84-9982-ff84c2538382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:17.288492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:17.310399 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1139/2271] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:17.321499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b2de6-1dfd-4726-aa1f-d99ca5fe31e3 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:17.323241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-964b2de6-1dfd-4726-aa1f-d99ca5fe31e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:17.379448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b2de6-1dfd-4726-aa1f-d99ca5fe31e3 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:17.383140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca3bd7-314f-42f8-b723-84b2d61f0788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:17.383791 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3bd7-314f-42f8-b723-84b2d61f0788 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:17.384082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3bd7-314f-42f8-b723-84b2d61f0788 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:17.384356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca3bd7-314f-42f8-b723-84b2d61f0788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:17.395894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca3bd7-314f-42f8-b723-84b2d61f0788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:17.397387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca3bd7-314f-42f8-b723-84b2d61f0788 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:17.407011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1133/2272] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:17.752363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f84f9-44f0-4c84-9982-ff84c2538382 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:17.762627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f84f9-44f0-4c84-9982-ff84c2538382 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:17.768644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec88c29-6e49-4e51-aff3-a1cd544a6b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:17.770077 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec88c29-6e49-4e51-aff3-a1cd544a6b13 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:17.770658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec88c29-6e49-4e51-aff3-a1cd544a6b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:17.771094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec88c29-6e49-4e51-aff3-a1cd544a6b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:17.773011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7ec88c29-6e49-4e51-aff3-a1cd544a6b13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:17.847049 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec88c29-6e49-4e51-aff3-a1cd544a6b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:17.848536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec88c29-6e49-4e51-aff3-a1cd544a6b13 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:17.857463 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1140/2273] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:17.985976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca3bd7-314f-42f8-b723-84b2d61f0788 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:18.052735 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca3bd7-314f-42f8-b723-84b2d61f0788 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:18.058709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:18.059583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:18.059863 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:18.060263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] PATH_INFO: `/v3/users/b979a489e0f94b35b99c7e874eeece46` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:18.061503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b979a489e0f94b35b99c7e874eeece46)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:18.063412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:18.133016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b979a489e0f94b35b99c7e874eeece46'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.133016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b979a489e0f94b35b99c7e874eeece46'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.138068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b979a489e0f94b35b99c7e874eeece46'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.138839 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b979a489e0f94b35b99c7e874eeece46'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.148412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b979a489e0f94b35b99c7e874eeece46'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.155148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:18.173459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1134/2274] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:17 2022] DELETE /identity/v3/users/b979a489e0f94b35b99c7e874eeece46 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:18.178607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f48b0f-375c-464e-a9cf-9d28368873a6 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:18.179035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f48b0f-375c-464e-a9cf-9d28368873a6 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:18.179474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f48b0f-375c-464e-a9cf-9d28368873a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:18.199924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-63f48b0f-375c-464e-a9cf-9d28368873a6 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:18.201305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-63f48b0f-375c-464e-a9cf-9d28368873a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:18.279464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1141/2275] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:07:18.290126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:18.291799 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:18.384302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f773d-17b7-4b14-8f90-cb0b97a1a24f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:18.393348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db602f-0fcc-4420-b993-91adef5c68ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:18.393348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db602f-0fcc-4420-b993-91adef5c68ee service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:18.393348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db602f-0fcc-4420-b993-91adef5c68ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:18.393348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db602f-0fcc-4420-b993-91adef5c68ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:18.404120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db602f-0fcc-4420-b993-91adef5c68ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:18.405960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db602f-0fcc-4420-b993-91adef5c68ee service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:18.416959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1135/2276] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:18.476633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f48b0f-375c-464e-a9cf-9d28368873a6 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:18.478204 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-63f48b0f-375c-464e-a9cf-9d28368873a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:18.542491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f48b0f-375c-464e-a9cf-9d28368873a6 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:18.546121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdaee90-7870-46d3-8825-8dfa36cc8676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:18.546792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaee90-7870-46d3-8825-8dfa36cc8676 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:18.547079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaee90-7870-46d3-8825-8dfa36cc8676 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:18.547359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaee90-7870-46d3-8825-8dfa36cc8676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:18.549470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-cbdaee90-7870-46d3-8825-8dfa36cc8676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:18.600624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:18.624437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdaee90-7870-46d3-8825-8dfa36cc8676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:18.625906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdaee90-7870-46d3-8825-8dfa36cc8676 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:18.635987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1142/2277] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:18.697055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db602f-0fcc-4420-b993-91adef5c68ee service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:18.698481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b4db602f-0fcc-4420-b993-91adef5c68ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:18.720767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db602f-0fcc-4420-b993-91adef5c68ee service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:18.727557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:18.728924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:18.729448 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:18.729922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] PATH_INFO: `/v3/projects/b1bd78079b2a4d49b2c3533111621164` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:18.731490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1bd78079b2a4d49b2c3533111621164)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:18.735079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:18.788921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1bd78079b2a4d49b2c3533111621164', 'request_id': 'req-9d27ac4e-b6c5-426a-98ca-764889c9c300'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.794908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1bd78079b2a4d49b2c3533111621164', 'request_id': 'req-9d27ac4e-b6c5-426a-98ca-764889c9c300'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.801857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1bd78079b2a4d49b2c3533111621164', 'request_id': 'req-9d27ac4e-b6c5-426a-98ca-764889c9c300'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.811978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] The token cache is being invalidate because project b1bd78079b2a4d49b2c3533111621164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:18.812262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.812914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.814458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1136/2278] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:18 2022] DELETE /identity/v3/projects/b1bd78079b2a4d49b2c3533111621164 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:18.819023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdaee90-7870-46d3-8825-8dfa36cc8676 service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:18.829725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdaee90-7870-46d3-8825-8dfa36cc8676 service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:18.888061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:18.889112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:18.889347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:18.889627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] PATH_INFO: `/v3/users/6421931f70604b6a89bc868ef771bf2a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:18.891024 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6421931f70604b6a89bc868ef771bf2a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:18.893150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:18.956592 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6421931f70604b6a89bc868ef771bf2a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.957589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6421931f70604b6a89bc868ef771bf2a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.964966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6421931f70604b6a89bc868ef771bf2a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.965771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6421931f70604b6a89bc868ef771bf2a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.979596 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2267011e-c222-48a8-8668-dfe687d903c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6421931f70604b6a89bc868ef771bf2a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:18.992597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1143/2279] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:18 2022] DELETE /identity/v3/users/6421931f70604b6a89bc868ef771bf2a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:19.290535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:19.291978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:19.308632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27ac4e-b6c5-426a-98ca-764889c9c300 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:19.382785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:19.384233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:19.384836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:19.385349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] PATH_INFO: `/v3/projects/2e5d2ab9db994fda8b0455cf5b486803` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:19.387626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e5d2ab9db994fda8b0455cf5b486803)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:19.390769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:19.394997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:19.449815 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e5d2ab9db994fda8b0455cf5b486803', 'request_id': 'req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:19.456017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e5d2ab9db994fda8b0455cf5b486803', 'request_id': 'req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:19.465918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e5d2ab9db994fda8b0455cf5b486803', 'request_id': 'req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:19.476759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] The token cache is being invalidate because project 2e5d2ab9db994fda8b0455cf5b486803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:19.477101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:19.477761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-d7e4776a-63e4-49b4-b0e0-993e03e7b6f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:19.479316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1137/2280] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:19 2022] DELETE /identity/v3/projects/2e5d2ab9db994fda8b0455cf5b486803 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:19.484266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2e627-b5cf-4960-900e-34e6b00cbe30 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:19.484588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2e627-b5cf-4960-900e-34e6b00cbe30 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:19.484869 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2e627-b5cf-4960-900e-34e6b00cbe30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:19.504611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b2b2e627-b5cf-4960-900e-34e6b00cbe30 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:19.505426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b2b2e627-b5cf-4960-900e-34e6b00cbe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:19.565337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:19.581129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1144/2281] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:19.589044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889a72c-094a-4cdb-b2c6-507284204212 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:19.589333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889a72c-094a-4cdb-b2c6-507284204212 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:19.589615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889a72c-094a-4cdb-b2c6-507284204212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:19.612485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6889a72c-094a-4cdb-b2c6-507284204212 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:19.613744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6889a72c-094a-4cdb-b2c6-507284204212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:19.624432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1138/2282] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:19.626970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2e627-b5cf-4960-900e-34e6b00cbe30 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:19.637777 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2e627-b5cf-4960-900e-34e6b00cbe30 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:19.642367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ad936-4691-4e81-b4f8-2c4d5136b71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:19.643030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ad936-4691-4e81-b4f8-2c4d5136b71c admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:19.643321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ad936-4691-4e81-b4f8-2c4d5136b71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:19.643603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ad936-4691-4e81-b4f8-2c4d5136b71c admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:19.644506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ad936-4691-4e81-b4f8-2c4d5136b71c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:07:19.644821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ad936-4691-4e81-b4f8-2c4d5136b71c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:19.646445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ad936-4691-4e81-b4f8-2c4d5136b71c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:19.660532 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1145/2283] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:07:19.663162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889a72c-094a-4cdb-b2c6-507284204212 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:19.672194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889a72c-094a-4cdb-b2c6-507284204212 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:19.677434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fe0e8-1403-42e5-bf2a-7c8de240135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:19.678094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fe0e8-1403-42e5-bf2a-7c8de240135f admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:19.678376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fe0e8-1403-42e5-bf2a-7c8de240135f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:19.678651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fe0e8-1403-42e5-bf2a-7c8de240135f admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:19.679337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fe0e8-1403-42e5-bf2a-7c8de240135f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:19.681285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fe0e8-1403-42e5-bf2a-7c8de240135f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:19.708141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1139/2284] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:19 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:19.712530 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ad936-4691-4e81-b4f8-2c4d5136b71c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:19.723822 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ad936-4691-4e81-b4f8-2c4d5136b71c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:19.792728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311481e-5562-4e5e-813a-58a952599e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:19.793341 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311481e-5562-4e5e-813a-58a952599e5a admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:19.793619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311481e-5562-4e5e-813a-58a952599e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:19.793891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311481e-5562-4e5e-813a-58a952599e5a admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:19.794608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311481e-5562-4e5e-813a-58a952599e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:19.796725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311481e-5562-4e5e-813a-58a952599e5a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:19.828619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1146/2285] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:19 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:07:19.833015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fe0e8-1403-42e5-bf2a-7c8de240135f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:19.843375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fe0e8-1403-42e5-bf2a-7c8de240135f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:19.848272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c664c76-8d53-4408-a8bb-a158a9287982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:19.849119 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c664c76-8d53-4408-a8bb-a158a9287982 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:19.849400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c664c76-8d53-4408-a8bb-a158a9287982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:19.849678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c664c76-8d53-4408-a8bb-a158a9287982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:19.850437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c664c76-8d53-4408-a8bb-a158a9287982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:19.852031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c664c76-8d53-4408-a8bb-a158a9287982 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:19.864234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1140/2286] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:07:19.867991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311481e-5562-4e5e-813a-58a952599e5a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:19.878825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311481e-5562-4e5e-813a-58a952599e5a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:19.884458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1d108-a3bf-410a-a320-70daf175641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:19.885112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1d108-a3bf-410a-a320-70daf175641b admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:19.885394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1d108-a3bf-410a-a320-70daf175641b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:19.885666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1d108-a3bf-410a-a320-70daf175641b admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:19.886398 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1d108-a3bf-410a-a320-70daf175641b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:19.887984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1d108-a3bf-410a-a320-70daf175641b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:19.900415 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1147/2287] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:19.904585 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c664c76-8d53-4408-a8bb-a158a9287982 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:19.915287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c664c76-8d53-4408-a8bb-a158a9287982 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:19.919980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143443e-161e-4a58-a24d-caf984739ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:19.920909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143443e-161e-4a58-a24d-caf984739ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:19.921617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143443e-161e-4a58-a24d-caf984739ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:19.921617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143443e-161e-4a58-a24d-caf984739ab8 admin admin] PATH_INFO: `/v3/projects/0c844c2b1fea403192a0b9c18504233e/users/6cfcd0790a8e4bfcb216fcd0868f8c10/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:19.938907 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143443e-161e-4a58-a24d-caf984739ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c844c2b1fea403192a0b9c18504233e, user_id=6cfcd0790a8e4bfcb216fcd0868f8c10, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:19.942561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143443e-161e-4a58-a24d-caf984739ab8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:19.959182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1141/2288] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:19 2022] PUT /identity/v3/projects/0c844c2b1fea403192a0b9c18504233e/users/6cfcd0790a8e4bfcb216fcd0868f8c10/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:19.970334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702c6eb-a318-49db-89e8-9e6b421b4957 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:19.970631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702c6eb-a318-49db-89e8-9e6b421b4957 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:19.970914 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702c6eb-a318-49db-89e8-9e6b421b4957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:20.000366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-4702c6eb-a318-49db-89e8-9e6b421b4957 None None] MFA Rules not processed for user `6cfcd0790a8e4bfcb216fcd0868f8c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:20.005871 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4702c6eb-a318-49db-89e8-9e6b421b4957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:20.095737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1148/2289] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:07:20.123518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143443e-161e-4a58-a24d-caf984739ab8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:20.125997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3143443e-161e-4a58-a24d-caf984739ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:20.198117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143443e-161e-4a58-a24d-caf984739ab8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:20.202351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d89c8-e355-41da-a7e7-fa2b8a452a3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:20.203132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d89c8-e355-41da-a7e7-fa2b8a452a3a service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:20.203418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d89c8-e355-41da-a7e7-fa2b8a452a3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:20.203762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d89c8-e355-41da-a7e7-fa2b8a452a3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:20.213375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d89c8-e355-41da-a7e7-fa2b8a452a3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:20.214869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d89c8-e355-41da-a7e7-fa2b8a452a3a service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:20.226447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1142/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:07:25.756930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702c6eb-a318-49db-89e8-9e6b421b4957 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:25.758506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4702c6eb-a318-49db-89e8-9e6b421b4957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:25.818030 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:25.845969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702c6eb-a318-49db-89e8-9e6b421b4957 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:25.852385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:25.853425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:25.853941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:25.854093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca145856-be31-40b8-90e0-2861064abff6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:25.854541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] PATH_INFO: `/v3/users/554816a509b249aaa920cdce385466b6` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:25.854636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca145856-be31-40b8-90e0-2861064abff6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:25.854762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca145856-be31-40b8-90e0-2861064abff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:25.856095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=554816a509b249aaa920cdce385466b6)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:25.858190 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:25.876304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ca145856-be31-40b8-90e0-2861064abff6 None None] MFA Rules not processed for user `cc19addacb9c412cb008e243c1fc7ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:25.877449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ca145856-be31-40b8-90e0-2861064abff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:25.910872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:25.934364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '554816a509b249aaa920cdce385466b6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:25.935130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '554816a509b249aaa920cdce385466b6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:25.943493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '554816a509b249aaa920cdce385466b6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:25.944139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '554816a509b249aaa920cdce385466b6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:25.958139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1143/2291] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:25.958834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '554816a509b249aaa920cdce385466b6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:25.976018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1149/2292] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:25 2022] DELETE /identity/v3/users/554816a509b249aaa920cdce385466b6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:25.981458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ce972-601c-4a57-8e36-ad5e18e9d270 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:25.981789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ce972-601c-4a57-8e36-ad5e18e9d270 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:25.982103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ce972-601c-4a57-8e36-ad5e18e9d270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:26.002017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8b0ce972-601c-4a57-8e36-ad5e18e9d270 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:26.003906 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8b0ce972-601c-4a57-8e36-ad5e18e9d270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:26.107818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1144/2293] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:26.119576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:26.122877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:26.196179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39140997-caa2-4956-845a-d8ed9a48f2eb admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:26.200726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418425e-f5b8-4b52-8bf5-1589e29d7ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:26.201488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418425e-f5b8-4b52-8bf5-1589e29d7ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:26.201826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418425e-f5b8-4b52-8bf5-1589e29d7ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:26.202182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418425e-f5b8-4b52-8bf5-1589e29d7ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:26.212409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418425e-f5b8-4b52-8bf5-1589e29d7ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:26.213969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418425e-f5b8-4b52-8bf5-1589e29d7ce4 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:26.229995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1150/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:26.557030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ce972-601c-4a57-8e36-ad5e18e9d270 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:26.558748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8b0ce972-601c-4a57-8e36-ad5e18e9d270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:26.575102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ce972-601c-4a57-8e36-ad5e18e9d270 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:26.579902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:26.580888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:26.581282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:26.581620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] PATH_INFO: `/v3/projects/745b66b72c1d49aca57c58f83111f348` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:26.583140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=745b66b72c1d49aca57c58f83111f348)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:26.585514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:26.655251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '745b66b72c1d49aca57c58f83111f348', 'request_id': 'req-4b69349b-30f1-4cfa-8a59-f07ecd80e185'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:26.663050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '745b66b72c1d49aca57c58f83111f348', 'request_id': 'req-4b69349b-30f1-4cfa-8a59-f07ecd80e185'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:26.671950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '745b66b72c1d49aca57c58f83111f348', 'request_id': 'req-4b69349b-30f1-4cfa-8a59-f07ecd80e185'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:26.684280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] The token cache is being invalidate because project 745b66b72c1d49aca57c58f83111f348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:26.684807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:26.685509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:26.687348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1145/2295] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:26 2022] DELETE /identity/v3/projects/745b66b72c1d49aca57c58f83111f348 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:26.692241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418425e-f5b8-4b52-8bf5-1589e29d7ce4 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:26.703727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418425e-f5b8-4b52-8bf5-1589e29d7ce4 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:26.772992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:26.774136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:26.774418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:26.774687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] PATH_INFO: `/v3/users/90f6c868c26a450f8c54d57600fbcb1d` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:26.776035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f6c868c26a450f8c54d57600fbcb1d)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:26.777852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:26.792524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:26.831416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f6c868c26a450f8c54d57600fbcb1d'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:26.832114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f6c868c26a450f8c54d57600fbcb1d'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:26.840824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f6c868c26a450f8c54d57600fbcb1d'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:26.841507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f6c868c26a450f8c54d57600fbcb1d'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:26.858474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f6c868c26a450f8c54d57600fbcb1d'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:26.873864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1151/2296] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:26 2022] DELETE /identity/v3/users/90f6c868c26a450f8c54d57600fbcb1d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:27.203740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:27.205845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:27.224908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69349b-30f1-4cfa-8a59-f07ecd80e185 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:27.288676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:27.289548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:27.289861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:27.290146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] PATH_INFO: `/v3/projects/a850ace674e341db952db6f3a541779e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:27.291363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a850ace674e341db952db6f3a541779e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:27.293349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:27.294810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:27.341986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a850ace674e341db952db6f3a541779e', 'request_id': 'req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:27.349283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a850ace674e341db952db6f3a541779e', 'request_id': 'req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:27.357664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a850ace674e341db952db6f3a541779e', 'request_id': 'req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:27.367991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] The token cache is being invalidate because project a850ace674e341db952db6f3a541779e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:27.368278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:27.368971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:27.370506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1146/2297] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:27 2022] DELETE /identity/v3/projects/a850ace674e341db952db6f3a541779e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:27.374621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:27.376012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:27.391936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c544995-3751-4dde-89f9-29a76cce8513 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:27.451533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:27.452423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:27.452717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:27.452992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] PATH_INFO: `/v3/users/c29563c76e644976a82c4035cf0db547` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:27.454171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29563c76e644976a82c4035cf0db547)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:27.456701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:27.512673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29563c76e644976a82c4035cf0db547'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:27.514143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29563c76e644976a82c4035cf0db547'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:27.524492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29563c76e644976a82c4035cf0db547'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:27.525330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29563c76e644976a82c4035cf0db547'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:27.538656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8df3e1a9-7677-40e9-a968-cfb73e4e0b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29563c76e644976a82c4035cf0db547'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:27.558288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1152/2298] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:27 2022] DELETE /identity/v3/users/c29563c76e644976a82c4035cf0db547 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:28.126507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:28.128080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:28.146007 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35618a8-cc11-41cf-a87c-bbf05ac6a86a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:28.223316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:28.224294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.224676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.225037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] PATH_INFO: `/v3/projects/8e37344d771444b2ac119e4a03e50bb6` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.226346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e37344d771444b2ac119e4a03e50bb6)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:28.228423 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:28.283527 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e37344d771444b2ac119e4a03e50bb6', 'request_id': 'req-857f7094-9dcf-4929-af7a-ab21a08aaed5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:28.290853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e37344d771444b2ac119e4a03e50bb6', 'request_id': 'req-857f7094-9dcf-4929-af7a-ab21a08aaed5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:28.299712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e37344d771444b2ac119e4a03e50bb6', 'request_id': 'req-857f7094-9dcf-4929-af7a-ab21a08aaed5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:28.310885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] The token cache is being invalidate because project 8e37344d771444b2ac119e4a03e50bb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:28.311173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:28.311817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-857f7094-9dcf-4929-af7a-ab21a08aaed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:28.313396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1147/2299] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:28 2022] DELETE /identity/v3/projects/8e37344d771444b2ac119e4a03e50bb6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:28.318432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce1c7e-e6f2-4484-b744-d26490e39f6e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.318934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce1c7e-e6f2-4484-b744-d26490e39f6e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.319029 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce1c7e-e6f2-4484-b744-d26490e39f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.336768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:28.350905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-f5ce1c7e-e6f2-4484-b744-d26490e39f6e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:28.352459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f5ce1c7e-e6f2-4484-b744-d26490e39f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:28.413810 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1153/2300] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:28.422010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc4d7e-69a8-4b47-bbbd-0c9804816508 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.422805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc4d7e-69a8-4b47-bbbd-0c9804816508 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.423091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc4d7e-69a8-4b47-bbbd-0c9804816508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.443331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1ddc4d7e-69a8-4b47-bbbd-0c9804816508 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:28.444558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1ddc4d7e-69a8-4b47-bbbd-0c9804816508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:28.452616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1148/2301] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:28.457331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce1c7e-e6f2-4484-b744-d26490e39f6e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:28.466910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce1c7e-e6f2-4484-b744-d26490e39f6e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:28.471338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744315b-839b-4efd-9150-13ef71ddc414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:28.472051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744315b-839b-4efd-9150-13ef71ddc414 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.472333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744315b-839b-4efd-9150-13ef71ddc414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.472634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744315b-839b-4efd-9150-13ef71ddc414 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.473369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744315b-839b-4efd-9150-13ef71ddc414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:07:28.473670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744315b-839b-4efd-9150-13ef71ddc414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:28.475274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744315b-839b-4efd-9150-13ef71ddc414 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:28.490208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1154/2302] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:07:28.491424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc4d7e-69a8-4b47-bbbd-0c9804816508 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:28.500612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc4d7e-69a8-4b47-bbbd-0c9804816508 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:28.506678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22880fb6-2302-4f8b-9a84-a81161270b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:28.507331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22880fb6-2302-4f8b-9a84-a81161270b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.507608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22880fb6-2302-4f8b-9a84-a81161270b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.507878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22880fb6-2302-4f8b-9a84-a81161270b8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.508596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22880fb6-2302-4f8b-9a84-a81161270b8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:28.510481 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22880fb6-2302-4f8b-9a84-a81161270b8a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:28.521232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:28.538708 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1149/2303] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:28 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:28.543036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744315b-839b-4efd-9150-13ef71ddc414 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:28.553206 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744315b-839b-4efd-9150-13ef71ddc414 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:28.623311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71927d-f16f-474f-9951-ce4b48110a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:28.624148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71927d-f16f-474f-9951-ce4b48110a1c admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.624449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71927d-f16f-474f-9951-ce4b48110a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.624724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71927d-f16f-474f-9951-ce4b48110a1c admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.625411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71927d-f16f-474f-9951-ce4b48110a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:28.628573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71927d-f16f-474f-9951-ce4b48110a1c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:28.649107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1155/2304] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:28 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:28.653205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22880fb6-2302-4f8b-9a84-a81161270b8a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:28.665057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22880fb6-2302-4f8b-9a84-a81161270b8a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:28.669788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f757890-25d8-4b15-ad93-7ff03fa8ef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:28.670472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f757890-25d8-4b15-ad93-7ff03fa8ef27 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.671369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f757890-25d8-4b15-ad93-7ff03fa8ef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.671644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f757890-25d8-4b15-ad93-7ff03fa8ef27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.672546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f757890-25d8-4b15-ad93-7ff03fa8ef27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:28.675077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f757890-25d8-4b15-ad93-7ff03fa8ef27 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:28.687171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1150/2305] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:28.691703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71927d-f16f-474f-9951-ce4b48110a1c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:28.701715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71927d-f16f-474f-9951-ce4b48110a1c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:28.706725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd28d01-2a75-4f5b-90fe-3e0f85666c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:28.707482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd28d01-2a75-4f5b-90fe-3e0f85666c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.707766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd28d01-2a75-4f5b-90fe-3e0f85666c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.708043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd28d01-2a75-4f5b-90fe-3e0f85666c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.708813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd28d01-2a75-4f5b-90fe-3e0f85666c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:28.710421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd28d01-2a75-4f5b-90fe-3e0f85666c74 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:28.721010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1156/2306] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:28.726095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f757890-25d8-4b15-ad93-7ff03fa8ef27 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:28.740473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f757890-25d8-4b15-ad93-7ff03fa8ef27 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:28.745569 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a822-05ef-4bcd-ae2a-57d982a23782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:28.746488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a822-05ef-4bcd-ae2a-57d982a23782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.746770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a822-05ef-4bcd-ae2a-57d982a23782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.747040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a822-05ef-4bcd-ae2a-57d982a23782 admin admin] PATH_INFO: `/v3/projects/89372ff5f7d74130a8e475a132d43d5f/users/2e25098e86f8438c9dfbaa36ae9b3fe3/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.762338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01a822-05ef-4bcd-ae2a-57d982a23782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89372ff5f7d74130a8e475a132d43d5f, user_id=2e25098e86f8438c9dfbaa36ae9b3fe3, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:28.765043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01a822-05ef-4bcd-ae2a-57d982a23782 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:28.778325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1151/2307] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:28 2022] PUT /identity/v3/projects/89372ff5f7d74130a8e475a132d43d5f/users/2e25098e86f8438c9dfbaa36ae9b3fe3/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:28.790070 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ba7e3-1d31-405a-b761-2d0d6ccef65e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.790362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ba7e3-1d31-405a-b761-2d0d6ccef65e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.790638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ba7e3-1d31-405a-b761-2d0d6ccef65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.817653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-5d4ba7e3-1d31-405a-b761-2d0d6ccef65e None None] MFA Rules not processed for user `2e25098e86f8438c9dfbaa36ae9b3fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:28.818904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5d4ba7e3-1d31-405a-b761-2d0d6ccef65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:28.877296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:28.884122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1157/2308] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:28.895360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a822-05ef-4bcd-ae2a-57d982a23782 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:28.897081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2a01a822-05ef-4bcd-ae2a-57d982a23782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:28.962243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a822-05ef-4bcd-ae2a-57d982a23782 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:28.967032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695a0cc-b810-4aad-9d52-2c4f37e762f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:28.967813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695a0cc-b810-4aad-9d52-2c4f37e762f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:28.968344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695a0cc-b810-4aad-9d52-2c4f37e762f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:28.968757 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695a0cc-b810-4aad-9d52-2c4f37e762f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:28.979006 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695a0cc-b810-4aad-9d52-2c4f37e762f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:28.980578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695a0cc-b810-4aad-9d52-2c4f37e762f8 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:28.992998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1152/2309] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:33.714815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ba7e3-1d31-405a-b761-2d0d6ccef65e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:33.783797 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ba7e3-1d31-405a-b761-2d0d6ccef65e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:33.788557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:33.789433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:33.789719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:33.790112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] PATH_INFO: `/v3/users/2e25098e86f8438c9dfbaa36ae9b3fe3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:33.791299 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e25098e86f8438c9dfbaa36ae9b3fe3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:33.793261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:33.843476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e25098e86f8438c9dfbaa36ae9b3fe3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:33.844163 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e25098e86f8438c9dfbaa36ae9b3fe3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:33.852678 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e25098e86f8438c9dfbaa36ae9b3fe3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:33.853402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e25098e86f8438c9dfbaa36ae9b3fe3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:33.866706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e25098e86f8438c9dfbaa36ae9b3fe3'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:33.881296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1158/2310] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:33 2022] DELETE /identity/v3/users/2e25098e86f8438c9dfbaa36ae9b3fe3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:33.885832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e82c12-3e05-479b-ab6f-166459e2dc1c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:33.886321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e82c12-3e05-479b-ab6f-166459e2dc1c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:33.886419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e82c12-3e05-479b-ab6f-166459e2dc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:33.904578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-73e82c12-3e05-479b-ab6f-166459e2dc1c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:33.905808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-73e82c12-3e05-479b-ab6f-166459e2dc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:33.972982 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1153/2311] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:33.982547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:33.984027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:34.041217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546bff7-7714-48ef-a074-c40d87f2c452 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:34.044868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebd51f-e817-4eb8-a426-10c75224270d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:34.045506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebd51f-e817-4eb8-a426-10c75224270d service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.045794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebd51f-e817-4eb8-a426-10c75224270d service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.046074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebd51f-e817-4eb8-a426-10c75224270d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.055692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebd51f-e817-4eb8-a426-10c75224270d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:34.057292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebd51f-e817-4eb8-a426-10c75224270d service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:34.067373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1159/2312] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:07:34.380028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e82c12-3e05-479b-ab6f-166459e2dc1c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:34.383124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-73e82c12-3e05-479b-ab6f-166459e2dc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:34.406699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e82c12-3e05-479b-ab6f-166459e2dc1c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:34.417838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:34.418789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.419679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.419679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] PATH_INFO: `/v3/projects/89372ff5f7d74130a8e475a132d43d5f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.421492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89372ff5f7d74130a8e475a132d43d5f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:34.423793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:34.473095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89372ff5f7d74130a8e475a132d43d5f', 'request_id': 'req-00ec202d-7507-416f-ba2a-5d27983fa025'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:34.479149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89372ff5f7d74130a8e475a132d43d5f', 'request_id': 'req-00ec202d-7507-416f-ba2a-5d27983fa025'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:34.480861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:34.487102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89372ff5f7d74130a8e475a132d43d5f', 'request_id': 'req-00ec202d-7507-416f-ba2a-5d27983fa025'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:34.503441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] The token cache is being invalidate because project 89372ff5f7d74130a8e475a132d43d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:34.503923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:34.504988 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-00ec202d-7507-416f-ba2a-5d27983fa025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:34.506969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1154/2313] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:34 2022] DELETE /identity/v3/projects/89372ff5f7d74130a8e475a132d43d5f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:34.514646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aad104-9053-4a38-8c2c-4266a681c079 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.514943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aad104-9053-4a38-8c2c-4266a681c079 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.515255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aad104-9053-4a38-8c2c-4266a681c079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.537989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-92aad104-9053-4a38-8c2c-4266a681c079 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:34.539224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-92aad104-9053-4a38-8c2c-4266a681c079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:34.554048 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:34.603923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1160/2314] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:34.611630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457b2aa-588b-4574-8b07-43720e1be562 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.612826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457b2aa-588b-4574-8b07-43720e1be562 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.613119 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457b2aa-588b-4574-8b07-43720e1be562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.631185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d457b2aa-588b-4574-8b07-43720e1be562 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:34.632426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d457b2aa-588b-4574-8b07-43720e1be562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:34.640261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1155/2315] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:34.644377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aad104-9053-4a38-8c2c-4266a681c079 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:34.653904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aad104-9053-4a38-8c2c-4266a681c079 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:34.658488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109309a-b2b4-42ed-9484-60135eb688c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:34.660083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109309a-b2b4-42ed-9484-60135eb688c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.660369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109309a-b2b4-42ed-9484-60135eb688c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.660688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109309a-b2b4-42ed-9484-60135eb688c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.661417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109309a-b2b4-42ed-9484-60135eb688c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:07:34.661726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109309a-b2b4-42ed-9484-60135eb688c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:34.663334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109309a-b2b4-42ed-9484-60135eb688c9 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:34.678095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1161/2316] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:07:34.679529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457b2aa-588b-4574-8b07-43720e1be562 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:34.688700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457b2aa-588b-4574-8b07-43720e1be562 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:34.693328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e707ae-83f4-4757-a42e-bab50079437d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:34.693966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e707ae-83f4-4757-a42e-bab50079437d admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.694245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e707ae-83f4-4757-a42e-bab50079437d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.694518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e707ae-83f4-4757-a42e-bab50079437d admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.695222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e707ae-83f4-4757-a42e-bab50079437d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:34.697152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e707ae-83f4-4757-a42e-bab50079437d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:34.722711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1156/2317] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:34 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:07:34.726784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109309a-b2b4-42ed-9484-60135eb688c9 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:34.739258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109309a-b2b4-42ed-9484-60135eb688c9 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:34.798531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb12f3-d268-4742-a463-248355ad96a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:34.799200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb12f3-d268-4742-a463-248355ad96a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.799906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb12f3-d268-4742-a463-248355ad96a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.800041 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb12f3-d268-4742-a463-248355ad96a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.800713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb12f3-d268-4742-a463-248355ad96a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:34.802639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb12f3-d268-4742-a463-248355ad96a2 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:34.826103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1162/2318] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:34 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:34.828104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e707ae-83f4-4757-a42e-bab50079437d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:34.839693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e707ae-83f4-4757-a42e-bab50079437d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:34.844280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9375e-a50f-49f8-a5a4-fc10fc341f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:34.844967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9375e-a50f-49f8-a5a4-fc10fc341f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.845250 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9375e-a50f-49f8-a5a4-fc10fc341f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.845524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9375e-a50f-49f8-a5a4-fc10fc341f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.846911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9375e-a50f-49f8-a5a4-fc10fc341f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:34.848953 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9375e-a50f-49f8-a5a4-fc10fc341f91 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:34.859679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1157/2319] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:34.863792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb12f3-d268-4742-a463-248355ad96a2 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:34.873733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb12f3-d268-4742-a463-248355ad96a2 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:34.878473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b73adc-a310-4aed-9fb4-05a97077b388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:34.879126 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b73adc-a310-4aed-9fb4-05a97077b388 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.879413 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b73adc-a310-4aed-9fb4-05a97077b388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.879691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b73adc-a310-4aed-9fb4-05a97077b388 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.880491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b73adc-a310-4aed-9fb4-05a97077b388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:34.882083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b73adc-a310-4aed-9fb4-05a97077b388 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:34.891925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1163/2320] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:34.896450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9375e-a50f-49f8-a5a4-fc10fc341f91 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:34.906384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9375e-a50f-49f8-a5a4-fc10fc341f91 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:34.912489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d3c36-f5ba-4978-ae3d-292f74134afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:34.913383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d3c36-f5ba-4978-ae3d-292f74134afc admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.913670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d3c36-f5ba-4978-ae3d-292f74134afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.913955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d3c36-f5ba-4978-ae3d-292f74134afc admin admin] PATH_INFO: `/v3/projects/43e5d438bb1f4eb28d3f3f68d77b5dee/users/00c126e7436243559b25981f5ca7c74e/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.929101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d3c36-f5ba-4978-ae3d-292f74134afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e5d438bb1f4eb28d3f3f68d77b5dee, user_id=00c126e7436243559b25981f5ca7c74e, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:34.931753 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d3c36-f5ba-4978-ae3d-292f74134afc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:34.946044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1158/2321] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:34 2022] PUT /identity/v3/projects/43e5d438bb1f4eb28d3f3f68d77b5dee/users/00c126e7436243559b25981f5ca7c74e/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:34.951398 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce4bf2-ae17-479c-a2d7-9a89d5215603 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:34.951790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce4bf2-ae17-479c-a2d7-9a89d5215603 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:34.952079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce4bf2-ae17-479c-a2d7-9a89d5215603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:34.970376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-82ce4bf2-ae17-479c-a2d7-9a89d5215603 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:34.972761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-82ce4bf2-ae17-479c-a2d7-9a89d5215603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:35.034367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1164/2322] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:35.043947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d3c36-f5ba-4978-ae3d-292f74134afc admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:35.046281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e28d3c36-f5ba-4978-ae3d-292f74134afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:35.105208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d3c36-f5ba-4978-ae3d-292f74134afc admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:35.108883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858a691-202d-4c0f-8ddd-ff1e0d19a4a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:35.109528 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a691-202d-4c0f-8ddd-ff1e0d19a4a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:35.109810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a691-202d-4c0f-8ddd-ff1e0d19a4a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:35.110086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a691-202d-4c0f-8ddd-ff1e0d19a4a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:35.122847 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858a691-202d-4c0f-8ddd-ff1e0d19a4a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:35.124686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858a691-202d-4c0f-8ddd-ff1e0d19a4a9 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:35.137467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1159/2323] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:07:39.095549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce4bf2-ae17-479c-a2d7-9a89d5215603 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:39.098263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-82ce4bf2-ae17-479c-a2d7-9a89d5215603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:39.146585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce4bf2-ae17-479c-a2d7-9a89d5215603 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:39.161305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:39.164918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:39.165473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:39.165792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] PATH_INFO: `/v3/users/afce14721a694cafbba342b6ef76b635` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:39.169639 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afce14721a694cafbba342b6ef76b635)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:39.177007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:39.234042 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:39.292287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afce14721a694cafbba342b6ef76b635'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:39.292962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afce14721a694cafbba342b6ef76b635'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:39.306637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afce14721a694cafbba342b6ef76b635'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:39.308729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afce14721a694cafbba342b6ef76b635'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:39.331672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afce14721a694cafbba342b6ef76b635'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:39.355336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1165/2324] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:07:39 2022] DELETE /identity/v3/users/afce14721a694cafbba342b6ef76b635 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:39.359576 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f661a-776a-44df-aa4b-b29f416c7e92 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:39.360068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f661a-776a-44df-aa4b-b29f416c7e92 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:39.360834 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f661a-776a-44df-aa4b-b29f416c7e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:39.382855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-999f661a-776a-44df-aa4b-b29f416c7e92 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:39.384352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-999f661a-776a-44df-aa4b-b29f416c7e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:39.457108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1160/2325] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:39.467255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:39.469414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:39.551296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8fdb8-2c84-478e-be2c-74c39e21c1c1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:39.555449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453860e4-fde1-426b-925f-6a0948f412be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:39.556237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453860e4-fde1-426b-925f-6a0948f412be service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:39.556877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453860e4-fde1-426b-925f-6a0948f412be service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:39.556877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453860e4-fde1-426b-925f-6a0948f412be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:39.569335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453860e4-fde1-426b-925f-6a0948f412be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:39.571012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453860e4-fde1-426b-925f-6a0948f412be service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:39.583447 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1166/2326] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:40.346252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f661a-776a-44df-aa4b-b29f416c7e92 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:40.348764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-999f661a-776a-44df-aa4b-b29f416c7e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:40.371858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f661a-776a-44df-aa4b-b29f416c7e92 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:40.387454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:40.388825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:40.389302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:40.389979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] PATH_INFO: `/v3/projects/78323d93142d481f812c5108ce58d5f1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:40.392236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=78323d93142d481f812c5108ce58d5f1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:40.396221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:40.441609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:40.494861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78323d93142d481f812c5108ce58d5f1', 'request_id': 'req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:40.507486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78323d93142d481f812c5108ce58d5f1', 'request_id': 'req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:40.526489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78323d93142d481f812c5108ce58d5f1', 'request_id': 'req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:40.556559 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] The token cache is being invalidate because project 78323d93142d481f812c5108ce58d5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:07:40.556959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:40.557636 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7219d9a6-4a6b-4c0a-83f1-0f43991890ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:07:40.562842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1161/2327] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:07:40 2022] DELETE /identity/v3/projects/78323d93142d481f812c5108ce58d5f1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:40.564261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9e2b5-0119-4172-889a-8adb8e53a487 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:40.565258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9e2b5-0119-4172-889a-8adb8e53a487 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:40.565690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9e2b5-0119-4172-889a-8adb8e53a487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:40.611954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6991-53c6-4caa-b012-a42c444172a8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:40.612353 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6991-53c6-4caa-b012-a42c444172a8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:40.612716 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d6991-53c6-4caa-b012-a42c444172a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:40.636253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e0d9e2b5-0119-4172-889a-8adb8e53a487 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:40.644072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e0d9e2b5-0119-4172-889a-8adb8e53a487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:40.670213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2a8d6991-53c6-4caa-b012-a42c444172a8 None None] MFA Rules not processed for user `00c126e7436243559b25981f5ca7c74e`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:40.671664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2a8d6991-53c6-4caa-b012-a42c444172a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:40.783455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1167/2328] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:40.785981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05fda2-cbc2-40e7-a590-a6549c6172bb None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:40.786275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05fda2-cbc2-40e7-a590-a6549c6172bb None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:40.786560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05fda2-cbc2-40e7-a590-a6549c6172bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:40.795400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1162/2329] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:40.802307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d6991-53c6-4caa-b012-a42c444172a8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:40.804991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2a8d6991-53c6-4caa-b012-a42c444172a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:40.825803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d6991-53c6-4caa-b012-a42c444172a8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:40.832320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64bd89-f6e6-4689-96ba-e81924c4c9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:40.833274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64bd89-f6e6-4689-96ba-e81924c4c9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:40.833562 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64bd89-f6e6-4689-96ba-e81924c4c9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:40.833842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64bd89-f6e6-4689-96ba-e81924c4c9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:40.834570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64bd89-f6e6-4689-96ba-e81924c4c9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:40.836513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64bd89-f6e6-4689-96ba-e81924c4c9b2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:40.837310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ed05fda2-cbc2-40e7-a590-a6549c6172bb None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:40.839153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ed05fda2-cbc2-40e7-a590-a6549c6172bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:40.849303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1168/2330] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:40.852842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05fda2-cbc2-40e7-a590-a6549c6172bb None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:40.869037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05fda2-cbc2-40e7-a590-a6549c6172bb None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:40.871673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1163/2331] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:40 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:40.880045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64bd89-f6e6-4689-96ba-e81924c4c9b2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:40.932624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:40.982684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412b1e6-f615-42fc-afb6-85a574480b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:40.983376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412b1e6-f615-42fc-afb6-85a574480b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:40.983657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412b1e6-f615-42fc-afb6-85a574480b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:40.984011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412b1e6-f615-42fc-afb6-85a574480b5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:40.984948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412b1e6-f615-42fc-afb6-85a574480b5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:07:40.985269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412b1e6-f615-42fc-afb6-85a574480b5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:40.987490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412b1e6-f615-42fc-afb6-85a574480b5d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:40.988819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64bd89-f6e6-4689-96ba-e81924c4c9b2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:40.994443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3470ee2-88dd-41d7-9d7e-7ac8e87c55e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:40.995338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3470ee2-88dd-41d7-9d7e-7ac8e87c55e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:40.995630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3470ee2-88dd-41d7-9d7e-7ac8e87c55e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:40.995902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3470ee2-88dd-41d7-9d7e-7ac8e87c55e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:40.996841 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3470ee2-88dd-41d7-9d7e-7ac8e87c55e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:40.998894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3470ee2-88dd-41d7-9d7e-7ac8e87c55e8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.005188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1169/2332] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 02:07:41.009788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412b1e6-f615-42fc-afb6-85a574480b5d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:41.025077 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412b1e6-f615-42fc-afb6-85a574480b5d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:41.032804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1164/2333] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:40 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:07:41.034054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3470ee2-88dd-41d7-9d7e-7ac8e87c55e8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:41.037391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cf7c3-b2d7-42fb-a8c3-ac71f53f3464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:41.038766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf7c3-b2d7-42fb-a8c3-ac71f53f3464 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.039564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf7c3-b2d7-42fb-a8c3-ac71f53f3464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.039942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf7c3-b2d7-42fb-a8c3-ac71f53f3464 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.040221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:41.040886 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cf7c3-b2d7-42fb-a8c3-ac71f53f3464 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:41.043208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cf7c3-b2d7-42fb-a8c3-ac71f53f3464 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.046192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3470ee2-88dd-41d7-9d7e-7ac8e87c55e8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:41.053496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff79165-62ec-4384-bbd4-305d58f64d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:41.054596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff79165-62ec-4384-bbd4-305d58f64d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.055202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff79165-62ec-4384-bbd4-305d58f64d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.055791 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff79165-62ec-4384-bbd4-305d58f64d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.056901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff79165-62ec-4384-bbd4-305d58f64d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:41.061165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff79165-62ec-4384-bbd4-305d58f64d29 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.072685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1165/2334] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:41.076815 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff79165-62ec-4384-bbd4-305d58f64d29 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:41.083426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1170/2335] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:07:41 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:41.084501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cf7c3-b2d7-42fb-a8c3-ac71f53f3464 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:41.089115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff79165-62ec-4384-bbd4-305d58f64d29 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:41.170809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c3a8d-53f3-4b14-9123-4b3b772e7eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:41.171578 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cf7c3-b2d7-42fb-a8c3-ac71f53f3464 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:41.171772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c3a8d-53f3-4b14-9123-4b3b772e7eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.171999 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c3a8d-53f3-4b14-9123-4b3b772e7eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.172275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c3a8d-53f3-4b14-9123-4b3b772e7eb7 admin admin] PATH_INFO: `/v3/projects/ba73173cd15a45f7a29d6b5112a33770/users/4828765f878648bdacb251e41bdf21a0/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.181512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb7e1f-fd2e-466d-838e-d3d49f54992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:41.182847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb7e1f-fd2e-466d-838e-d3d49f54992d admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.183260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb7e1f-fd2e-466d-838e-d3d49f54992d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.183756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb7e1f-fd2e-466d-838e-d3d49f54992d admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.184634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb7e1f-fd2e-466d-838e-d3d49f54992d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:41.200746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb7e1f-fd2e-466d-838e-d3d49f54992d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.213250 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c3a8d-53f3-4b14-9123-4b3b772e7eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba73173cd15a45f7a29d6b5112a33770, user_id=4828765f878648bdacb251e41bdf21a0, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:41.216181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c3a8d-53f3-4b14-9123-4b3b772e7eb7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.258487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1166/2336] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:41 2022] PUT /identity/v3/projects/ba73173cd15a45f7a29d6b5112a33770/users/4828765f878648bdacb251e41bdf21a0/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:07:41.262167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c3a8d-53f3-4b14-9123-4b3b772e7eb7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:41.269930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1171/2337] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:07:41 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:07:41.274450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb7e1f-fd2e-466d-838e-d3d49f54992d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:41.291879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb7e1f-fd2e-466d-838e-d3d49f54992d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:41.390259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f991b-8558-4e48-a2c7-ee5e40f233bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:41.391115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f991b-8558-4e48-a2c7-ee5e40f233bf admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.391473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f991b-8558-4e48-a2c7-ee5e40f233bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.391917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f991b-8558-4e48-a2c7-ee5e40f233bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.392838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f991b-8558-4e48-a2c7-ee5e40f233bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:41.393416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c3a8d-53f3-4b14-9123-4b3b772e7eb7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:41.395370 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f991b-8558-4e48-a2c7-ee5e40f233bf admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.415167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1172/2338] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:41.416487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f991b-8558-4e48-a2c7-ee5e40f233bf admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:41.417179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce477abe-c2aa-4ef9-877d-37414d08eeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:41.417881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce477abe-c2aa-4ef9-877d-37414d08eeda admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.418162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce477abe-c2aa-4ef9-877d-37414d08eeda admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.418433 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce477abe-c2aa-4ef9-877d-37414d08eeda admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.419176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce477abe-c2aa-4ef9-877d-37414d08eeda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:41.420826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce477abe-c2aa-4ef9-877d-37414d08eeda admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.429131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f991b-8558-4e48-a2c7-ee5e40f233bf admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:41.434972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ae7f9-24dc-4fad-91c4-c2d05da7194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:41.435804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ae7f9-24dc-4fad-91c4-c2d05da7194a admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.436222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ae7f9-24dc-4fad-91c4-c2d05da7194a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.436703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ae7f9-24dc-4fad-91c4-c2d05da7194a admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.437686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ae7f9-24dc-4fad-91c4-c2d05da7194a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:41.440406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ae7f9-24dc-4fad-91c4-c2d05da7194a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.447792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1167/2339] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:41.448486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce477abe-c2aa-4ef9-877d-37414d08eeda admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:41.463625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce477abe-c2aa-4ef9-877d-37414d08eeda admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:41.464569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1173/2340] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:07:41.468487 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ae7f9-24dc-4fad-91c4-c2d05da7194a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:41.473845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a981f2-7376-46f6-a3ec-52299d63c289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:41.474809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a981f2-7376-46f6-a3ec-52299d63c289 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.475105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a981f2-7376-46f6-a3ec-52299d63c289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.475382 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a981f2-7376-46f6-a3ec-52299d63c289 admin admin] PATH_INFO: `/v3/projects/ba73173cd15a45f7a29d6b5112a33770/users/4828765f878648bdacb251e41bdf21a0/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.477621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a981f2-7376-46f6-a3ec-52299d63c289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba73173cd15a45f7a29d6b5112a33770, user_id=4828765f878648bdacb251e41bdf21a0, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:41.481857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a981f2-7376-46f6-a3ec-52299d63c289 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.488279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ae7f9-24dc-4fad-91c4-c2d05da7194a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:41.499598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1168/2341] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:41 2022] PUT /identity/v3/projects/ba73173cd15a45f7a29d6b5112a33770/users/4828765f878648bdacb251e41bdf21a0/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:41.507389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a795f-8bdc-45a1-9989-53d10f636575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:41.508716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a795f-8bdc-45a1-9989-53d10f636575 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.509069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a795f-8bdc-45a1-9989-53d10f636575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.509516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a795f-8bdc-45a1-9989-53d10f636575 admin admin] PATH_INFO: `/v3/projects/68cd7ed883a04e91b36482a8950b12d1/users/f9556a3adc6b40ee93c498fb98cd77ea/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.529209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a795f-8bdc-45a1-9989-53d10f636575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68cd7ed883a04e91b36482a8950b12d1, user_id=f9556a3adc6b40ee93c498fb98cd77ea, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:41.532563 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a795f-8bdc-45a1-9989-53d10f636575 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.543763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:41.552433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1174/2342] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:07:41 2022] PUT /identity/v3/projects/68cd7ed883a04e91b36482a8950b12d1/users/f9556a3adc6b40ee93c498fb98cd77ea/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:07:41.557089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15eab26-ee97-4366-9f83-f676eb974756 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.557453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15eab26-ee97-4366-9f83-f676eb974756 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.557682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15eab26-ee97-4366-9f83-f676eb974756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.583091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e15eab26-ee97-4366-9f83-f676eb974756 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:41.584342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e15eab26-ee97-4366-9f83-f676eb974756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:41.670266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:41.688881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1169/2343] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:07:41.700901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a795f-8bdc-45a1-9989-53d10f636575 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:41.702851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c46a795f-8bdc-45a1-9989-53d10f636575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:41.807953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a795f-8bdc-45a1-9989-53d10f636575 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:41.812708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f915097-588e-4ac5-9be5-ea7031a43242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:41.813909 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f915097-588e-4ac5-9be5-ea7031a43242 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:41.814304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f915097-588e-4ac5-9be5-ea7031a43242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:41.814908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f915097-588e-4ac5-9be5-ea7031a43242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:41.831211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f915097-588e-4ac5-9be5-ea7031a43242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:41.837714 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f915097-588e-4ac5-9be5-ea7031a43242 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:41.863563 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1175/2344] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:42.859340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971aafd2-c693-4bba-b63a-71a870f38c91 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:42.859744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971aafd2-c693-4bba-b63a-71a870f38c91 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:42.859893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971aafd2-c693-4bba-b63a-71a870f38c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:42.921536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-971aafd2-c693-4bba-b63a-71a870f38c91 None None] MFA Rules not processed for user `f9556a3adc6b40ee93c498fb98cd77ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:42.922793 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-971aafd2-c693-4bba-b63a-71a870f38c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:43.078095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1170/2345] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:43.094093 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f915097-588e-4ac5-9be5-ea7031a43242 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:43.098637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7f915097-588e-4ac5-9be5-ea7031a43242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:43.184374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f915097-588e-4ac5-9be5-ea7031a43242 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:43.189274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593adf7-f608-49b6-ae49-0ff1edcb1982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:43.190659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593adf7-f608-49b6-ae49-0ff1edcb1982 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:43.190921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593adf7-f608-49b6-ae49-0ff1edcb1982 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:43.191303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593adf7-f608-49b6-ae49-0ff1edcb1982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:43.206107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593adf7-f608-49b6-ae49-0ff1edcb1982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:43.207747 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593adf7-f608-49b6-ae49-0ff1edcb1982 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:43.235520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1176/2346] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:07:44.813053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f926-8e2b-4d0d-8846-3b54826237d9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:44.814064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f926-8e2b-4d0d-8846-3b54826237d9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:44.814669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f926-8e2b-4d0d-8846-3b54826237d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:44.837879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a770f926-8e2b-4d0d-8846-3b54826237d9 None None] MFA Rules not processed for user `f9556a3adc6b40ee93c498fb98cd77ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:44.839176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a770f926-8e2b-4d0d-8846-3b54826237d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:44.846692 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1171/2347] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:50.305584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29c727-f713-4d9b-9be1-43ec207b37d2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:50.306541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29c727-f713-4d9b-9be1-43ec207b37d2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:50.307204 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29c727-f713-4d9b-9be1-43ec207b37d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:50.349869 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-8f29c727-f713-4d9b-9be1-43ec207b37d2 None None] MFA Rules not processed for user `4828765f878648bdacb251e41bdf21a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:50.352036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8f29c727-f713-4d9b-9be1-43ec207b37d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:50.524772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1177/2348] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:50 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:50.531717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f926-8e2b-4d0d-8846-3b54826237d9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:50.535110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a770f926-8e2b-4d0d-8846-3b54826237d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:50.810118 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f926-8e2b-4d0d-8846-3b54826237d9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:50.817694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238d130-ce00-480f-a923-5768f29b01a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:50.818750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238d130-ce00-480f-a923-5768f29b01a5 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:50.819140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238d130-ce00-480f-a923-5768f29b01a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:50.819547 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238d130-ce00-480f-a923-5768f29b01a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:50.966575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:50.980754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238d130-ce00-480f-a923-5768f29b01a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:50.982960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238d130-ce00-480f-a923-5768f29b01a5 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:51.009442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1172/2349] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:07:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:07:51.846767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160f53c-572f-41ee-b799-3009db50ab4e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:51.847753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160f53c-572f-41ee-b799-3009db50ab4e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:51.848093 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160f53c-572f-41ee-b799-3009db50ab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:51.871152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3160f53c-572f-41ee-b799-3009db50ab4e None None] MFA Rules not processed for user `9e13bd970d2e4d67b62452ae0d6cae56`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:51.872764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3160f53c-572f-41ee-b799-3009db50ab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:51.914672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:07:52.006006 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1178/2350] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:53.510879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb760b-1dce-41f5-93e7-0aac104db052 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:53.511375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb760b-1dce-41f5-93e7-0aac104db052 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:53.511534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb760b-1dce-41f5-93e7-0aac104db052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:53.538297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0dcb760b-1dce-41f5-93e7-0aac104db052 None None] MFA Rules not processed for user `00c126e7436243559b25981f5ca7c74e`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:53.539518 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0dcb760b-1dce-41f5-93e7-0aac104db052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:53.553977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1173/2351] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:07:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:56.382092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af814d-6c62-4787-99e3-676636c39995 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:56.382579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af814d-6c62-4787-99e3-676636c39995 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:56.382667 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af814d-6c62-4787-99e3-676636c39995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:56.413483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-d8af814d-6c62-4787-99e3-676636c39995 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:07:56.414627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d8af814d-6c62-4787-99e3-676636c39995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:56.473585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1179/2352] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:07:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:07:56.515672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb760b-1dce-41f5-93e7-0aac104db052 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:07:56.517665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0dcb760b-1dce-41f5-93e7-0aac104db052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:07:56.589852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb760b-1dce-41f5-93e7-0aac104db052 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:07:56.596231 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea1bf1-5d9b-4e22-a128-ab68bd50cb20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:07:56.596920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1bf1-5d9b-4e22-a128-ab68bd50cb20 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:07:56.597200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1bf1-5d9b-4e22-a128-ab68bd50cb20 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:07:56.597472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea1bf1-5d9b-4e22-a128-ab68bd50cb20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:07:56.605842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea1bf1-5d9b-4e22-a128-ab68bd50cb20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:07:56.607453 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea1bf1-5d9b-4e22-a128-ab68bd50cb20 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:07:56.616264 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1174/2353] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:07:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:08:05.646933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af814d-6c62-4787-99e3-676636c39995 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:05.648834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d8af814d-6c62-4787-99e3-676636c39995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:05.670564 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af814d-6c62-4787-99e3-676636c39995 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:05.676571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:05.677482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:05.677770 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:05.678045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] PATH_INFO: `/v3/users/6cfcd0790a8e4bfcb216fcd0868f8c10` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:05.679295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cfcd0790a8e4bfcb216fcd0868f8c10)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:05.681301 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:05.732559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cfcd0790a8e4bfcb216fcd0868f8c10'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:05.733197 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cfcd0790a8e4bfcb216fcd0868f8c10'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:05.743293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cfcd0790a8e4bfcb216fcd0868f8c10'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:05.743948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cfcd0790a8e4bfcb216fcd0868f8c10'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:05.759505 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cfcd0790a8e4bfcb216fcd0868f8c10'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:05.780465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1180/2354] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:08:05 2022] DELETE /identity/v3/users/6cfcd0790a8e4bfcb216fcd0868f8c10 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:08:05.782523 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cf95e-d547-48ad-93a0-9b8555653b2d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:05.783428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cf95e-d547-48ad-93a0-9b8555653b2d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:05.783935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cf95e-d547-48ad-93a0-9b8555653b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:05.804965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-032cf95e-d547-48ad-93a0-9b8555653b2d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:08:05.806454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-032cf95e-d547-48ad-93a0-9b8555653b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:05.873972 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1175/2355] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:08:05.884315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:05.885826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:05.949317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:05.971977 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1613b4e-a1c9-44bc-8f32-1ddc68cedcad admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:05.977380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6add63-4fb5-43c8-98c6-ac7a134ea2f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:05.978145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6add63-4fb5-43c8-98c6-ac7a134ea2f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:05.978441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6add63-4fb5-43c8-98c6-ac7a134ea2f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:05.978764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6add63-4fb5-43c8-98c6-ac7a134ea2f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:05.990542 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6add63-4fb5-43c8-98c6-ac7a134ea2f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:05.992115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6add63-4fb5-43c8-98c6-ac7a134ea2f4 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:06.007232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1181/2356] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:08:06.441693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cf95e-d547-48ad-93a0-9b8555653b2d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:06.443671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-032cf95e-d547-48ad-93a0-9b8555653b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:06.458195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cf95e-d547-48ad-93a0-9b8555653b2d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:06.463335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:06.464319 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:06.464695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:06.465045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] PATH_INFO: `/v3/projects/0c844c2b1fea403192a0b9c18504233e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:06.466332 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c844c2b1fea403192a0b9c18504233e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:06.468557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:06.530383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c844c2b1fea403192a0b9c18504233e', 'request_id': 'req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:06.538495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c844c2b1fea403192a0b9c18504233e', 'request_id': 'req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:06.547261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c844c2b1fea403192a0b9c18504233e', 'request_id': 'req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:06.554110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:06.561119 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] The token cache is being invalidate because project 0c844c2b1fea403192a0b9c18504233e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:08:06.561516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:06.562957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:06.565182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1176/2357] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:08:06 2022] DELETE /identity/v3/projects/0c844c2b1fea403192a0b9c18504233e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:08:16.001258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=7 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:16.581870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:23.961051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29357ea6-0183-4214-a63b-96b68f8d6fdf None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:23.961565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29357ea6-0183-4214-a63b-96b68f8d6fdf None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:23.961664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29357ea6-0183-4214-a63b-96b68f8d6fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:24.011557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-29357ea6-0183-4214-a63b-96b68f8d6fdf None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:08:24.012821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-29357ea6-0183-4214-a63b-96b68f8d6fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:24.083957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1182/2358] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jun 8 02:08:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:08:24.097689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:24.100156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:24.144275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29357ea6-0183-4214-a63b-96b68f8d6fdf None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:24.145707 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-29357ea6-0183-4214-a63b-96b68f8d6fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:24.199811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00bd08-777f-4e18-8fa2-76b3cfc75e6e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:24.205235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02286d-699d-43ab-b5ac-986fdad4b2e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:24.205894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02286d-699d-43ab-b5ac-986fdad4b2e7 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:24.206180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02286d-699d-43ab-b5ac-986fdad4b2e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:24.206458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02286d-699d-43ab-b5ac-986fdad4b2e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:24.215281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02286d-699d-43ab-b5ac-986fdad4b2e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:24.216916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02286d-699d-43ab-b5ac-986fdad4b2e7 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:24.225948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1177/2359] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:08:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:08:24.243015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29357ea6-0183-4214-a63b-96b68f8d6fdf None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:24.248210 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a484f3-8b14-4d1b-8f81-6f834cc51ade service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:24.248901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a484f3-8b14-4d1b-8f81-6f834cc51ade service placement] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:24.249192 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a484f3-8b14-4d1b-8f81-6f834cc51ade service placement] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:24.249474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a484f3-8b14-4d1b-8f81-6f834cc51ade service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:24.251633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e3a484f3-8b14-4d1b-8f81-6f834cc51ade service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:24.264013 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a484f3-8b14-4d1b-8f81-6f834cc51ade service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:24.265657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a484f3-8b14-4d1b-8f81-6f834cc51ade service placement] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:24.276646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1183/2360] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:08:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:08:26.170313 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02286d-699d-43ab-b5ac-986fdad4b2e7 service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:26.172525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ad02286d-699d-43ab-b5ac-986fdad4b2e7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:26.248115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02286d-699d-43ab-b5ac-986fdad4b2e7 service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:26.253005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:26.253876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:26.254154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:26.254431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] PATH_INFO: `/v3/users/cc19addacb9c412cb008e243c1fc7ace` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:26.255645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc19addacb9c412cb008e243c1fc7ace)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:26.258610 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:26.311217 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc19addacb9c412cb008e243c1fc7ace'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:26.311860 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc19addacb9c412cb008e243c1fc7ace'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:26.318642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc19addacb9c412cb008e243c1fc7ace'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:26.319909 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc19addacb9c412cb008e243c1fc7ace'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:26.327236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-5d7ce3a1-06ad-4cca-8eac-94c8b8e9accc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc19addacb9c412cb008e243c1fc7ace'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:26.335263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:26.347504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1178/2361] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:08:26 2022] DELETE /identity/v3/users/cc19addacb9c412cb008e243c1fc7ace => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:08:26.523547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a484f3-8b14-4d1b-8f81-6f834cc51ade service placement] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:26.534889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a484f3-8b14-4d1b-8f81-6f834cc51ade service placement] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:26.581089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:26.598743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:26.599617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:26.599906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:26.600226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] PATH_INFO: `/v3/projects/540c921edbf24dbeaf13675b583bb464` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:26.601466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=540c921edbf24dbeaf13675b583bb464)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:26.603463 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:26.650877 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '540c921edbf24dbeaf13675b583bb464', 'request_id': 'req-ad883b35-821c-44a6-aa63-7734d2129d9c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:26.656374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '540c921edbf24dbeaf13675b583bb464', 'request_id': 'req-ad883b35-821c-44a6-aa63-7734d2129d9c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:26.662764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '540c921edbf24dbeaf13675b583bb464', 'request_id': 'req-ad883b35-821c-44a6-aa63-7734d2129d9c'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:26.674344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] The token cache is being invalidate because project 540c921edbf24dbeaf13675b583bb464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:08:26.674653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:26.675269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ad883b35-821c-44a6-aa63-7734d2129d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:26.676826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1184/2362] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:08:26 2022] DELETE /identity/v3/projects/540c921edbf24dbeaf13675b583bb464 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:08:26.683431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5cfb9-51c5-40c6-8065-221c1e53f05a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:26.683728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5cfb9-51c5-40c6-8065-221c1e53f05a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:26.684010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5cfb9-51c5-40c6-8065-221c1e53f05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:26.702491 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-15b5cfb9-51c5-40c6-8065-221c1e53f05a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:08:26.703702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-15b5cfb9-51c5-40c6-8065-221c1e53f05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:26.766107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1179/2363] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:08:26.773673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b342183-095a-4221-8985-172d25ce96bf None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:26.773970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b342183-095a-4221-8985-172d25ce96bf None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:26.774257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b342183-095a-4221-8985-172d25ce96bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:26.793328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-2b342183-095a-4221-8985-172d25ce96bf None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:08:26.794682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2b342183-095a-4221-8985-172d25ce96bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:26.803277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1185/2364] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:08:26.808109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5cfb9-51c5-40c6-8065-221c1e53f05a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:26.818084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5cfb9-51c5-40c6-8065-221c1e53f05a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:26.823055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa87330-3e3a-4bf8-a62b-70a4e765867c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:26.823732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa87330-3e3a-4bf8-a62b-70a4e765867c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:26.824019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa87330-3e3a-4bf8-a62b-70a4e765867c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:26.824298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa87330-3e3a-4bf8-a62b-70a4e765867c admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:26.825061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa87330-3e3a-4bf8-a62b-70a4e765867c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:08:26.825373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa87330-3e3a-4bf8-a62b-70a4e765867c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:26.826979 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa87330-3e3a-4bf8-a62b-70a4e765867c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:26.839196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1180/2365] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:08:26.843478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b342183-095a-4221-8985-172d25ce96bf None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:26.853726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b342183-095a-4221-8985-172d25ce96bf None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:26.858769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a26f6-c290-4945-b976-b56f16c5072c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:26.859423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a26f6-c290-4945-b976-b56f16c5072c admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:26.859709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a26f6-c290-4945-b976-b56f16c5072c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:26.859987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a26f6-c290-4945-b976-b56f16c5072c admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:26.860716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a26f6-c290-4945-b976-b56f16c5072c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:26.862615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a26f6-c290-4945-b976-b56f16c5072c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:26.889217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1186/2366] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:08:26 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:08:26.893439 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa87330-3e3a-4bf8-a62b-70a4e765867c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:26.903611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa87330-3e3a-4bf8-a62b-70a4e765867c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:26.960483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341702e-f665-4e24-8d78-a53fdf319c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:26.961161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341702e-f665-4e24-8d78-a53fdf319c4a admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:26.961442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341702e-f665-4e24-8d78-a53fdf319c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:26.961717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341702e-f665-4e24-8d78-a53fdf319c4a admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:26.962407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341702e-f665-4e24-8d78-a53fdf319c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:26.964350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341702e-f665-4e24-8d78-a53fdf319c4a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:26.983856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1181/2367] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:08:26 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:08:26.988078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a26f6-c290-4945-b976-b56f16c5072c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:26.999104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a26f6-c290-4945-b976-b56f16c5072c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:27.003667 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5bb9b-c02e-4e36-a99b-1411c1d6ab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:27.004323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5bb9b-c02e-4e36-a99b-1411c1d6ab8b admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:27.004638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5bb9b-c02e-4e36-a99b-1411c1d6ab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:27.004942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5bb9b-c02e-4e36-a99b-1411c1d6ab8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:27.005665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5bb9b-c02e-4e36-a99b-1411c1d6ab8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:27.009513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5bb9b-c02e-4e36-a99b-1411c1d6ab8b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:27.018801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1187/2368] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:08:27.023114 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341702e-f665-4e24-8d78-a53fdf319c4a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:27.033173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341702e-f665-4e24-8d78-a53fdf319c4a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:27.038902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10d31a-70cd-4729-8600-dfc0d1698a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:27.039520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10d31a-70cd-4729-8600-dfc0d1698a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:27.039801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10d31a-70cd-4729-8600-dfc0d1698a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:27.040076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10d31a-70cd-4729-8600-dfc0d1698a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:27.040835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10d31a-70cd-4729-8600-dfc0d1698a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:27.042457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10d31a-70cd-4729-8600-dfc0d1698a26 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:27.052218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1182/2369] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:08:27.057089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5bb9b-c02e-4e36-a99b-1411c1d6ab8b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:27.069986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5bb9b-c02e-4e36-a99b-1411c1d6ab8b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:27.074715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebb6f1-2279-4d6a-8dcf-3d41b17d7d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:27.075589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebb6f1-2279-4d6a-8dcf-3d41b17d7d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:27.075868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebb6f1-2279-4d6a-8dcf-3d41b17d7d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:27.076139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebb6f1-2279-4d6a-8dcf-3d41b17d7d4e admin admin] PATH_INFO: `/v3/projects/2f79742baeab463fbefc5cf365d0e518/users/db9c89ff274f40caa5b1ba5fff35c34e/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:27.098250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebb6f1-2279-4d6a-8dcf-3d41b17d7d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f79742baeab463fbefc5cf365d0e518, user_id=db9c89ff274f40caa5b1ba5fff35c34e, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:27.101534 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebb6f1-2279-4d6a-8dcf-3d41b17d7d4e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:27.114028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1188/2370] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:08:27 2022] PUT /identity/v3/projects/2f79742baeab463fbefc5cf365d0e518/users/db9c89ff274f40caa5b1ba5fff35c34e/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:08:27.125512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4133c1-c251-4651-a867-a8c4010f2259 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:27.125804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4133c1-c251-4651-a867-a8c4010f2259 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:27.126088 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4133c1-c251-4651-a867-a8c4010f2259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:27.153236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3d4133c1-c251-4651-a867-a8c4010f2259 None None] MFA Rules not processed for user `db9c89ff274f40caa5b1ba5fff35c34e`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:08:27.154468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3d4133c1-c251-4651-a867-a8c4010f2259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:27.203208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:27.227371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1183/2371] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:08:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:08:27.238272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebb6f1-2279-4d6a-8dcf-3d41b17d7d4e admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:27.240008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8eebb6f1-2279-4d6a-8dcf-3d41b17d7d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:27.318689 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:27.322481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebb6f1-2279-4d6a-8dcf-3d41b17d7d4e admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:27.328750 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff60a3a-9e9d-40cb-8aa0-962da801170f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:27.329874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff60a3a-9e9d-40cb-8aa0-962da801170f service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:27.330200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff60a3a-9e9d-40cb-8aa0-962da801170f service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:27.330737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff60a3a-9e9d-40cb-8aa0-962da801170f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:27.344227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff60a3a-9e9d-40cb-8aa0-962da801170f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:27.345841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff60a3a-9e9d-40cb-8aa0-962da801170f service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:27.357278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1189/2372] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:08:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:08:27.387676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4133c1-c251-4651-a867-a8c4010f2259 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:27.461767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4133c1-c251-4651-a867-a8c4010f2259 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:27.468230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:27.468650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:27.468933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:27.469204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] PATH_INFO: `/v3/users/db9c89ff274f40caa5b1ba5fff35c34e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:27.470447 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9c89ff274f40caa5b1ba5fff35c34e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:27.472443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:27.529637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9c89ff274f40caa5b1ba5fff35c34e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:27.530352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9c89ff274f40caa5b1ba5fff35c34e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:27.538023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9c89ff274f40caa5b1ba5fff35c34e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:27.538662 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9c89ff274f40caa5b1ba5fff35c34e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:27.546233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9c89ff274f40caa5b1ba5fff35c34e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:27.564169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1184/2373] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:08:27 2022] DELETE /identity/v3/users/db9c89ff274f40caa5b1ba5fff35c34e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:08:27.569220 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62662dc0-2d40-4ef4-895e-e0d9e320c88e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:27.569677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62662dc0-2d40-4ef4-895e-e0d9e320c88e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:27.569821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62662dc0-2d40-4ef4-895e-e0d9e320c88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:27.587961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-62662dc0-2d40-4ef4-895e-e0d9e320c88e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:08:27.589221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-62662dc0-2d40-4ef4-895e-e0d9e320c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:27.649752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1190/2374] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:08:27.659738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:27.661149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:27.718748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd406f-370e-4412-b2e7-896f9236e7db admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:27.722683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238ff10-6482-419e-8ca7-83ac99ffbaa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:27.723376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238ff10-6482-419e-8ca7-83ac99ffbaa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:27.723746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238ff10-6482-419e-8ca7-83ac99ffbaa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:27.724023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238ff10-6482-419e-8ca7-83ac99ffbaa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:27.734320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238ff10-6482-419e-8ca7-83ac99ffbaa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:27.735763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238ff10-6482-419e-8ca7-83ac99ffbaa1 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:27.748220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1185/2375] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:08:28.017410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62662dc0-2d40-4ef4-895e-e0d9e320c88e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:28.018891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-62662dc0-2d40-4ef4-895e-e0d9e320c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:28.035130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62662dc0-2d40-4ef4-895e-e0d9e320c88e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:28.040503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:28.041573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:28.048760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:28.049091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] PATH_INFO: `/v3/projects/2f79742baeab463fbefc5cf365d0e518` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:28.050404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f79742baeab463fbefc5cf365d0e518)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:28.052374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:28.105979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f79742baeab463fbefc5cf365d0e518', 'request_id': 'req-893bf91e-6fb5-4c70-b5ed-859b697f35ed'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:28.111705 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f79742baeab463fbefc5cf365d0e518', 'request_id': 'req-893bf91e-6fb5-4c70-b5ed-859b697f35ed'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:28.119363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f79742baeab463fbefc5cf365d0e518', 'request_id': 'req-893bf91e-6fb5-4c70-b5ed-859b697f35ed'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:28.139933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] The token cache is being invalidate because project 2f79742baeab463fbefc5cf365d0e518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:08:28.140362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:28.141163 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:08:28.142890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1191/2376] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:08:28 2022] DELETE /identity/v3/projects/2f79742baeab463fbefc5cf365d0e518 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:08:35.409156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2f259-d94b-49a7-a114-cc45bdcb0631 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:35.409714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2f259-d94b-49a7-a114-cc45bdcb0631 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:35.409714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2f259-d94b-49a7-a114-cc45bdcb0631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:35.422835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:35.429947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6fa2f259-d94b-49a7-a114-cc45bdcb0631 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:08:35.431871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6fa2f259-d94b-49a7-a114-cc45bdcb0631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:35.493349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1186/2377] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:08:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:08:35.509110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:35.512628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:35.584896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bf91e-6fb5-4c70-b5ed-859b697f35ed admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:35.590834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6eee1a-56f9-436b-9963-4a00d9e72405 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:35.591540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eee1a-56f9-436b-9963-4a00d9e72405 service swift] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:35.591828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eee1a-56f9-436b-9963-4a00d9e72405 service swift] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:35.592108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eee1a-56f9-436b-9963-4a00d9e72405 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:35.601876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6eee1a-56f9-436b-9963-4a00d9e72405 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:35.603461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6eee1a-56f9-436b-9963-4a00d9e72405 service swift] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:35.611900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1192/2378] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:08:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:08:45.486162 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=22 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:45.609010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:08:49.549857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2f259-d94b-49a7-a114-cc45bdcb0631 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:49.551262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6fa2f259-d94b-49a7-a114-cc45bdcb0631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:49.606487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2f259-d94b-49a7-a114-cc45bdcb0631 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:49.610130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49b583-8792-4ec0-983e-bb85c2d9b26f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:49.610768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49b583-8792-4ec0-983e-bb85c2d9b26f service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:49.611050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49b583-8792-4ec0-983e-bb85c2d9b26f service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:49.611326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49b583-8792-4ec0-983e-bb85c2d9b26f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:49.613082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cb49b583-8792-4ec0-983e-bb85c2d9b26f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:49.676762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49b583-8792-4ec0-983e-bb85c2d9b26f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:49.678816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49b583-8792-4ec0-983e-bb85c2d9b26f service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:49.688470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1187/2379] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:08:49 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:08:58.505305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa2419-8372-4be7-b757-08b73c7a58b2 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:58.505905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa2419-8372-4be7-b757-08b73c7a58b2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:58.506187 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa2419-8372-4be7-b757-08b73c7a58b2 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:58.507557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1193/2380] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:08:58.514007 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da79b6-6aa8-45bd-8cf9-8d2946806128 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:58.514497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da79b6-6aa8-45bd-8cf9-8d2946806128 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:58.514598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da79b6-6aa8-45bd-8cf9-8d2946806128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:58.553208 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-39da79b6-6aa8-45bd-8cf9-8d2946806128 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:08:58.554466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-39da79b6-6aa8-45bd-8cf9-8d2946806128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:58.636508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1188/2381] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:08:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:08:58.649145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa2419-8372-4be7-b757-08b73c7a58b2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:58.650840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2efa2419-8372-4be7-b757-08b73c7a58b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:58.734433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa2419-8372-4be7-b757-08b73c7a58b2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:58.738600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612b992-f286-4583-84d7-81b30eecd6e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d1e01f6f82a4ebbb15d0cb55b963af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:58.739277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612b992-f286-4583-84d7-81b30eecd6e7 service glance] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:58.739559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612b992-f286-4583-84d7-81b30eecd6e7 service glance] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:58.739873 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612b992-f286-4583-84d7-81b30eecd6e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:58.753087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612b992-f286-4583-84d7-81b30eecd6e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:58.754524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612b992-f286-4583-84d7-81b30eecd6e7 service glance] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:58.765794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1194/2382] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:08:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:08:58.834544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd493b1-729b-4087-a25a-f50895898467 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:58.834893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd493b1-729b-4087-a25a-f50895898467 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:58.835130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd493b1-729b-4087-a25a-f50895898467 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:58.836566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1189/2383] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:08:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:08:58.841481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db97d5-e189-4a18-bf70-9a1492a2971d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:58.841798 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db97d5-e189-4a18-bf70-9a1492a2971d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:58.842076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db97d5-e189-4a18-bf70-9a1492a2971d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:58.860556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-09db97d5-e189-4a18-bf70-9a1492a2971d None None] MFA Rules not processed for user `d26f4426ba8c43399c4a7ef2f1701f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:08:58.861650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-09db97d5-e189-4a18-bf70-9a1492a2971d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:58.869768 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1195/2384] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:08:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:08:58.882546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd493b1-729b-4087-a25a-f50895898467 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:58.884436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7fd493b1-729b-4087-a25a-f50895898467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:58.897688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd493b1-729b-4087-a25a-f50895898467 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:58.901185 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a859c-56f5-4571-9821-f8e24e137fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:58.901832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a859c-56f5-4571-9821-f8e24e137fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:58.902110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a859c-56f5-4571-9821-f8e24e137fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:58.902382 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a859c-56f5-4571-9821-f8e24e137fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:58.912063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a859c-56f5-4571-9821-f8e24e137fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:58.914105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a859c-56f5-4571-9821-f8e24e137fd9 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:58.926969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1190/2385] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:08:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:08:59.015502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc2786-19b7-4996-8659-93cbc81a8d62 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:59.015866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc2786-19b7-4996-8659-93cbc81a8d62 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:59.016218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc2786-19b7-4996-8659-93cbc81a8d62 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:59.017661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1196/2386] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:08:59.023621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b3d54-7e3d-48c5-9535-ea873f9da699 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:59.023918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b3d54-7e3d-48c5-9535-ea873f9da699 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:59.024228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b3d54-7e3d-48c5-9535-ea873f9da699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:59.043582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-420b3d54-7e3d-48c5-9535-ea873f9da699 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:08:59.044810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-420b3d54-7e3d-48c5-9535-ea873f9da699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:08:59.054842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1191/2387] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:08:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:08:59.065051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc2786-19b7-4996-8659-93cbc81a8d62 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:08:59.076851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc2786-19b7-4996-8659-93cbc81a8d62 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:08:59.080616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e3f4c-b92d-4c5a-af54-d8a926ca2613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:08:59.081249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e3f4c-b92d-4c5a-af54-d8a926ca2613 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:08:59.081528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e3f4c-b92d-4c5a-af54-d8a926ca2613 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:08:59.081800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e3f4c-b92d-4c5a-af54-d8a926ca2613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:08:59.092280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e3f4c-b92d-4c5a-af54-d8a926ca2613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:08:59.093731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e3f4c-b92d-4c5a-af54-d8a926ca2613 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:08:59.104968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1197/2388] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:08:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:09:01.821456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca6fee-7337-4dcd-9018-787d472f5c17 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:09:01.821967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca6fee-7337-4dcd-9018-787d472f5c17 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:09:01.822072 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca6fee-7337-4dcd-9018-787d472f5c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:09:01.839543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d9ca6fee-7337-4dcd-9018-787d472f5c17 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:09:01.840798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d9ca6fee-7337-4dcd-9018-787d472f5c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:09:01.847905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1192/2389] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:09:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:09:01.869444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e3f4c-b92d-4c5a-af54-d8a926ca2613 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:09:01.881924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e3f4c-b92d-4c5a-af54-d8a926ca2613 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:09:01.888244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406bee-ce98-449c-9975-8722977cefe8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:09:01.889116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406bee-ce98-449c-9975-8722977cefe8 service swift] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:09:01.889405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406bee-ce98-449c-9975-8722977cefe8 service swift] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:09:01.889687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406bee-ce98-449c-9975-8722977cefe8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:09:01.898169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406bee-ce98-449c-9975-8722977cefe8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:09:01.899943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406bee-ce98-449c-9975-8722977cefe8 service swift] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:09:01.908799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1198/2390] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:09:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:09:11.834881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:09:11.904806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=50 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:09:27.644477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d544a8c-24af-4e1d-b7a8-bde829f4e3a1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:09:27.645018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d544a8c-24af-4e1d-b7a8-bde829f4e3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:09:27.645018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d544a8c-24af-4e1d-b7a8-bde829f4e3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:09:27.663362 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8d544a8c-24af-4e1d-b7a8-bde829f4e3a1 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:09:27.665103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8d544a8c-24af-4e1d-b7a8-bde829f4e3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:09:27.672756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1193/2391] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jun 8 02:09:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:09:27.685276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406bee-ce98-449c-9975-8722977cefe8 service swift] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:09:27.696539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406bee-ce98-449c-9975-8722977cefe8 service swift] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:09:27.702414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75bf51-9586-4c41-8885-23b87c15f81b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:09:27.703237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75bf51-9586-4c41-8885-23b87c15f81b service swift] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:09:27.703520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75bf51-9586-4c41-8885-23b87c15f81b service swift] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:09:27.703794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75bf51-9586-4c41-8885-23b87c15f81b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:09:27.713045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75bf51-9586-4c41-8885-23b87c15f81b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:09:27.715596 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75bf51-9586-4c41-8885-23b87c15f81b service swift] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:09:27.724176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1199/2392] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:09:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 08 02:09:27.879802 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b4b8e-ff41-48d5-944d-5b936b83b3e4 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:09:27.880134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b4b8e-ff41-48d5-944d-5b936b83b3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:09:27.880994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b4b8e-ff41-48d5-944d-5b936b83b3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:09:27.900801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-524b4b8e-ff41-48d5-944d-5b936b83b3e4 None None] MFA Rules not processed for user `f9556a3adc6b40ee93c498fb98cd77ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:09:27.902033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-524b4b8e-ff41-48d5-944d-5b936b83b3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:09:27.969335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1194/2393] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:09:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:09:37.720852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=6 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:09:37.961812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=28 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:09:47.658226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040a74-5c17-4994-a78d-6b3fb151510a None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:09:47.659203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040a74-5c17-4994-a78d-6b3fb151510a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:09:47.659501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040a74-5c17-4994-a78d-6b3fb151510a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:09:47.680582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-db040a74-5c17-4994-a78d-6b3fb151510a None None] MFA Rules not processed for user `9e13bd970d2e4d67b62452ae0d6cae56`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:09:47.682053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-db040a74-5c17-4994-a78d-6b3fb151510a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:09:47.739007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1200/2394] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:09:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:09:57.732302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=25 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:10:20.194493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b989bf2-f89f-4dad-a935-343b64aa4f5e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:20.195045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b989bf2-f89f-4dad-a935-343b64aa4f5e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:20.195045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b989bf2-f89f-4dad-a935-343b64aa4f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:20.214027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0b989bf2-f89f-4dad-a935-343b64aa4f5e None None] MFA Rules not processed for user `a3b4081bebe74c839f5aa214275210b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:10:20.215233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0b989bf2-f89f-4dad-a935-343b64aa4f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:20.271071 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1195/2395] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:10:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:10:22.053698 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db040a74-5c17-4994-a78d-6b3fb151510a None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:22.066048 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db040a74-5c17-4994-a78d-6b3fb151510a None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:22.072088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b48c9-1b49-41b1-88d3-df82f4079f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:22.073302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b48c9-1b49-41b1-88d3-df82f4079f2a service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:22.073968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b48c9-1b49-41b1-88d3-df82f4079f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:22.074375 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b48c9-1b49-41b1-88d3-df82f4079f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:22.076863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ed9b48c9-1b49-41b1-88d3-df82f4079f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:22.097243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b48c9-1b49-41b1-88d3-df82f4079f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:22.099661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b48c9-1b49-41b1-88d3-df82f4079f2a service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:22.116233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1201/2396] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:10:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:10:25.787095 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b989bf2-f89f-4dad-a935-343b64aa4f5e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:25.797934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b989bf2-f89f-4dad-a935-343b64aa4f5e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:25.801549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb8ce6-5fde-4f88-8b7d-cabaa6d33587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:25.802194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb8ce6-5fde-4f88-8b7d-cabaa6d33587 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:25.802478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb8ce6-5fde-4f88-8b7d-cabaa6d33587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:25.802942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb8ce6-5fde-4f88-8b7d-cabaa6d33587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:25.804750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4ecb8ce6-5fde-4f88-8b7d-cabaa6d33587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:25.818630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb8ce6-5fde-4f88-8b7d-cabaa6d33587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:25.820103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb8ce6-5fde-4f88-8b7d-cabaa6d33587 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:25.831699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1196/2397] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:10:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:10:28.209061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b48c9-1b49-41b1-88d3-df82f4079f2a service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:28.210582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ed9b48c9-1b49-41b1-88d3-df82f4079f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:28.279804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b48c9-1b49-41b1-88d3-df82f4079f2a service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:28.284677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:28.285766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:28.286058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:28.286418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] PATH_INFO: `/v3/users/f9556a3adc6b40ee93c498fb98cd77ea` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:28.287852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9556a3adc6b40ee93c498fb98cd77ea)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:28.290861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:28.343511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9556a3adc6b40ee93c498fb98cd77ea'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:10:28.344265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9556a3adc6b40ee93c498fb98cd77ea'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:10:28.351254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9556a3adc6b40ee93c498fb98cd77ea'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:10:28.351879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9556a3adc6b40ee93c498fb98cd77ea'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:10:28.364297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9556a3adc6b40ee93c498fb98cd77ea'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:10:28.377803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1202/2398] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:10:28 2022] DELETE /identity/v3/users/f9556a3adc6b40ee93c498fb98cd77ea => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:10:28.548189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb8ce6-5fde-4f88-8b7d-cabaa6d33587 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:28.558944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb8ce6-5fde-4f88-8b7d-cabaa6d33587 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:28.615592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:28.616482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:28.616764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:28.617037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] PATH_INFO: `/v3/projects/68cd7ed883a04e91b36482a8950b12d1` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:28.618238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] RBAC: Authorizing `identity:delete_project(project_id=68cd7ed883a04e91b36482a8950b12d1)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:28.620152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:28.665338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68cd7ed883a04e91b36482a8950b12d1', 'request_id': 'req-6ee57976-a8de-410f-9cec-f2a65445078b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:10:28.671206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68cd7ed883a04e91b36482a8950b12d1', 'request_id': 'req-6ee57976-a8de-410f-9cec-f2a65445078b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:10:28.678903 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68cd7ed883a04e91b36482a8950b12d1', 'request_id': 'req-6ee57976-a8de-410f-9cec-f2a65445078b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:10:28.689526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] The token cache is being invalidate because project 68cd7ed883a04e91b36482a8950b12d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:10:28.689810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:10:28.690424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6ee57976-a8de-410f-9cec-f2a65445078b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:10:28.691969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1197/2399] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:10:28 2022] DELETE /identity/v3/projects/68cd7ed883a04e91b36482a8950b12d1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:10:32.653888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:32.655520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:32.726141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:10:32.730777 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b1cd2-36e9-42d7-bf91-b2bc42af9c5e admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:32.797469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b7484-3dc7-4044-a20d-6e3618994cb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:32.798113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b7484-3dc7-4044-a20d-6e3618994cb6 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:32.798398 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b7484-3dc7-4044-a20d-6e3618994cb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:32.798672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b7484-3dc7-4044-a20d-6e3618994cb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:32.800483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-109b7484-3dc7-4044-a20d-6e3618994cb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:32.856927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b7484-3dc7-4044-a20d-6e3618994cb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:32.858524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b7484-3dc7-4044-a20d-6e3618994cb6 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:32.867642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1203/2400] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:10:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:10:34.917504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325c9a6-42a8-48dd-904c-431116787507 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:34.919421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325c9a6-42a8-48dd-904c-431116787507 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:34.919708 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325c9a6-42a8-48dd-904c-431116787507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:34.938401 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3325c9a6-42a8-48dd-904c-431116787507 None None] MFA Rules not processed for user `1b43d2b8b9084099b832332a565e741c`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:10:34.939626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3325c9a6-42a8-48dd-904c-431116787507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:34.947429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1198/2401] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:10:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:10:35.307091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b7484-3dc7-4044-a20d-6e3618994cb6 service nova] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:35.308720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-109b7484-3dc7-4044-a20d-6e3618994cb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:35.368267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b7484-3dc7-4044-a20d-6e3618994cb6 service nova] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:35.371913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf602e3-0770-481c-87af-eb4f751807d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:35.372614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf602e3-0770-481c-87af-eb4f751807d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:35.372906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf602e3-0770-481c-87af-eb4f751807d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:35.373189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf602e3-0770-481c-87af-eb4f751807d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:35.375027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-bdf602e3-0770-481c-87af-eb4f751807d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:35.385658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf602e3-0770-481c-87af-eb4f751807d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:35.387044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf602e3-0770-481c-87af-eb4f751807d2 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:35.395779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1204/2402] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:10:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:10:35.708903 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325c9a6-42a8-48dd-904c-431116787507 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:35.710325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3325c9a6-42a8-48dd-904c-431116787507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:35.731968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:10:35.787561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325c9a6-42a8-48dd-904c-431116787507 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:35.792658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d4fe4-78c9-4926-a9d0-a151800250d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:35.793306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4fe4-78c9-4926-a9d0-a151800250d2 service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:35.793591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4fe4-78c9-4926-a9d0-a151800250d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:35.793865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4fe4-78c9-4926-a9d0-a151800250d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:35.795622 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6d3d4fe4-78c9-4926-a9d0-a151800250d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:35.808209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d4fe4-78c9-4926-a9d0-a151800250d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:35.809857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d4fe4-78c9-4926-a9d0-a151800250d2 service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:35.821253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1199/2403] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:10:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:10:38.292790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf602e3-0770-481c-87af-eb4f751807d2 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:38.302258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf602e3-0770-481c-87af-eb4f751807d2 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:38.306584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c0bd1-ef4d-4716-aa1f-6ce0c995d2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:38.307242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c0bd1-ef4d-4716-aa1f-6ce0c995d2fe service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:38.307642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c0bd1-ef4d-4716-aa1f-6ce0c995d2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:38.308047 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c0bd1-ef4d-4716-aa1f-6ce0c995d2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:38.310139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8a4c0bd1-ef4d-4716-aa1f-6ce0c995d2fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:38.320837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c0bd1-ef4d-4716-aa1f-6ce0c995d2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:38.322302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c0bd1-ef4d-4716-aa1f-6ce0c995d2fe service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:38.331623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1205/2404] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:10:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:10:39.120309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d4fe4-78c9-4926-a9d0-a151800250d2 service placement] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:39.121855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6d3d4fe4-78c9-4926-a9d0-a151800250d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:39.180679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d4fe4-78c9-4926-a9d0-a151800250d2 service placement] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:39.184244 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c361912-9370-4cb0-b3a0-386df0336237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:39.184931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c361912-9370-4cb0-b3a0-386df0336237 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:39.185221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c361912-9370-4cb0-b3a0-386df0336237 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:39.185498 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c361912-9370-4cb0-b3a0-386df0336237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:39.187289 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8c361912-9370-4cb0-b3a0-386df0336237 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:39.249726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c361912-9370-4cb0-b3a0-386df0336237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:39.251386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c361912-9370-4cb0-b3a0-386df0336237 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:39.262303 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1200/2405] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:10:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:10:48.327637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=86 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:10:48.600928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c0bd1-ef4d-4716-aa1f-6ce0c995d2fe service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:48.603517 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8a4c0bd1-ef4d-4716-aa1f-6ce0c995d2fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:48.619471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c0bd1-ef4d-4716-aa1f-6ce0c995d2fe service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:48.625009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e71aff-1ef3-489b-b5b8-90f0e4c5cdf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:48.625654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e71aff-1ef3-489b-b5b8-90f0e4c5cdf0 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:48.625940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e71aff-1ef3-489b-b5b8-90f0e4c5cdf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:48.626221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e71aff-1ef3-489b-b5b8-90f0e4c5cdf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:48.627984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-08e71aff-1ef3-489b-b5b8-90f0e4c5cdf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:48.642724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e71aff-1ef3-489b-b5b8-90f0e4c5cdf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:48.644423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e71aff-1ef3-489b-b5b8-90f0e4c5cdf0 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:48.658489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1206/2406] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:10:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:10:49.256639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=73 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:10:58.451504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c361912-9370-4cb0-b3a0-386df0336237 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:10:58.460965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c361912-9370-4cb0-b3a0-386df0336237 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:10:58.464506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30618e21-4bdd-4998-b812-eef3fdec428a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:10:58.465142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30618e21-4bdd-4998-b812-eef3fdec428a service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:10:58.465419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30618e21-4bdd-4998-b812-eef3fdec428a service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:10:58.465688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30618e21-4bdd-4998-b812-eef3fdec428a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:10:58.467421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-30618e21-4bdd-4998-b812-eef3fdec428a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:10:58.530440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30618e21-4bdd-4998-b812-eef3fdec428a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:10:58.532136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30618e21-4bdd-4998-b812-eef3fdec428a service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:10:58.542411 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1201/2407] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:10:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:10:58.652631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=6 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:01.826280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e71aff-1ef3-489b-b5b8-90f0e4c5cdf0 service nova] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:01.827800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-08e71aff-1ef3-489b-b5b8-90f0e4c5cdf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:01.842543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e71aff-1ef3-489b-b5b8-90f0e4c5cdf0 service nova] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:01.847621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6784c-5e4d-4d20-9b3b-58afe60c15d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:01.848336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6784c-5e4d-4d20-9b3b-58afe60c15d5 service placement] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:01.848792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6784c-5e4d-4d20-9b3b-58afe60c15d5 service placement] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:01.849136 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6784c-5e4d-4d20-9b3b-58afe60c15d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:01.851673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f7a6784c-5e4d-4d20-9b3b-58afe60c15d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:01.864102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6784c-5e4d-4d20-9b3b-58afe60c15d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:01.865814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6784c-5e4d-4d20-9b3b-58afe60c15d5 service placement] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:01.877703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1207/2408] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:01.890149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30618e21-4bdd-4998-b812-eef3fdec428a service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:01.891709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-30618e21-4bdd-4998-b812-eef3fdec428a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:01.907196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30618e21-4bdd-4998-b812-eef3fdec428a service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:01.917701 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:01.918594 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:01.918882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:01.919154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] PATH_INFO: `/v3/users/9e13bd970d2e4d67b62452ae0d6cae56` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:01.920451 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e13bd970d2e4d67b62452ae0d6cae56)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:01.922356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:01.972027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e13bd970d2e4d67b62452ae0d6cae56'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:01.972695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e13bd970d2e4d67b62452ae0d6cae56'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:01.980321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e13bd970d2e4d67b62452ae0d6cae56'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:01.981046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e13bd970d2e4d67b62452ae0d6cae56'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:01.988131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e0225758-4f52-4d82-a7b9-adb2c68ff89d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e13bd970d2e4d67b62452ae0d6cae56'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:02.005357 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1202/2409] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:11:01 2022] DELETE /identity/v3/users/9e13bd970d2e4d67b62452ae0d6cae56 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:02.209720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6784c-5e4d-4d20-9b3b-58afe60c15d5 service placement] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:02.222168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6784c-5e4d-4d20-9b3b-58afe60c15d5 service placement] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:02.281315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:02.282429 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.282719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.282995 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] PATH_INFO: `/v3/projects/d40d753f66ec4ed2808aa65b825e66f8` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.284179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d40d753f66ec4ed2808aa65b825e66f8)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:02.286139 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:02.332430 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd40d753f66ec4ed2808aa65b825e66f8', 'request_id': 'req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:02.338046 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd40d753f66ec4ed2808aa65b825e66f8', 'request_id': 'req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:02.344113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd40d753f66ec4ed2808aa65b825e66f8', 'request_id': 'req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:02.356641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] The token cache is being invalidate because project d40d753f66ec4ed2808aa65b825e66f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:11:02.356930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:02.357593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7243e0ac-e7cc-4ba6-8e9f-de36ff88f901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:02.359168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1208/2410] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:11:02 2022] DELETE /identity/v3/projects/d40d753f66ec4ed2808aa65b825e66f8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:02.364138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8266-3464-49bf-9ab5-2d5ba919086d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.364468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8266-3464-49bf-9ab5-2d5ba919086d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.364756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8266-3464-49bf-9ab5-2d5ba919086d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.382717 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-919a8266-3464-49bf-9ab5-2d5ba919086d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:02.383972 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-919a8266-3464-49bf-9ab5-2d5ba919086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:02.444183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1203/2411] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:02.452010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bdf95-765a-4fae-a32f-5b734cb3a60f None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.452506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bdf95-765a-4fae-a32f-5b734cb3a60f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.453152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bdf95-765a-4fae-a32f-5b734cb3a60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.476027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e54bdf95-765a-4fae-a32f-5b734cb3a60f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:02.477249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e54bdf95-765a-4fae-a32f-5b734cb3a60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:02.485100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1209/2412] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:02.489318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a8266-3464-49bf-9ab5-2d5ba919086d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:02.499947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a8266-3464-49bf-9ab5-2d5ba919086d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:02.504516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325784d-365f-4d39-9532-390da2e33b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:02.505172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325784d-365f-4d39-9532-390da2e33b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.505460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325784d-365f-4d39-9532-390da2e33b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.505740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325784d-365f-4d39-9532-390da2e33b87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.506461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325784d-365f-4d39-9532-390da2e33b87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:11:02.506941 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325784d-365f-4d39-9532-390da2e33b87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:02.508701 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325784d-365f-4d39-9532-390da2e33b87 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:02.510157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:02.521605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1204/2413] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:11:02.525650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bdf95-765a-4fae-a32f-5b734cb3a60f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:02.535055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bdf95-765a-4fae-a32f-5b734cb3a60f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:02.540314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334a7c2-54af-4310-aec6-6548c6d4d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:02.540990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334a7c2-54af-4310-aec6-6548c6d4d1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.541274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334a7c2-54af-4310-aec6-6548c6d4d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.541553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334a7c2-54af-4310-aec6-6548c6d4d1a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.542248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334a7c2-54af-4310-aec6-6548c6d4d1a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:02.544156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334a7c2-54af-4310-aec6-6548c6d4d1a3 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:02.569562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1210/2414] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:11:02 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:11:02.573338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325784d-365f-4d39-9532-390da2e33b87 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:02.583306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325784d-365f-4d39-9532-390da2e33b87 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:02.648882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dea92-9671-4a0d-b8ed-ea2b4d301adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:02.649551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dea92-9671-4a0d-b8ed-ea2b4d301adc admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.649834 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dea92-9671-4a0d-b8ed-ea2b4d301adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.650121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dea92-9671-4a0d-b8ed-ea2b4d301adc admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.650798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dea92-9671-4a0d-b8ed-ea2b4d301adc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:02.652764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dea92-9671-4a0d-b8ed-ea2b4d301adc admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:02.672475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1205/2415] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:11:02 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:02.676477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334a7c2-54af-4310-aec6-6548c6d4d1a3 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:02.686852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334a7c2-54af-4310-aec6-6548c6d4d1a3 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:02.691828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483690e-f7a5-4202-81b8-8981ac43fc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:02.692522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483690e-f7a5-4202-81b8-8981ac43fc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.692812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483690e-f7a5-4202-81b8-8981ac43fc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.693088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483690e-f7a5-4202-81b8-8981ac43fc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.693821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483690e-f7a5-4202-81b8-8981ac43fc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:02.695579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483690e-f7a5-4202-81b8-8981ac43fc03 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:02.707814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1211/2416] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:11:02.712302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dea92-9671-4a0d-b8ed-ea2b4d301adc admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:02.722392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dea92-9671-4a0d-b8ed-ea2b4d301adc admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:02.727715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b56646-f6fd-4f01-b645-3480d691a2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:02.728384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b56646-f6fd-4f01-b645-3480d691a2bf admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.728698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b56646-f6fd-4f01-b645-3480d691a2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.728973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b56646-f6fd-4f01-b645-3480d691a2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.729696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b56646-f6fd-4f01-b645-3480d691a2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:02.731294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b56646-f6fd-4f01-b645-3480d691a2bf admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:02.741756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1206/2417] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:02.745948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483690e-f7a5-4202-81b8-8981ac43fc03 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:02.757482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483690e-f7a5-4202-81b8-8981ac43fc03 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:02.762333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4da27d-a304-46a0-8f84-297dda31c1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:02.763215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4da27d-a304-46a0-8f84-297dda31c1b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.763500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4da27d-a304-46a0-8f84-297dda31c1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.763772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4da27d-a304-46a0-8f84-297dda31c1b4 admin admin] PATH_INFO: `/v3/projects/489d21703ba940c1857622700a940ddc/users/05a3371f82b3480b9be0e0264585a678/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.780250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4da27d-a304-46a0-8f84-297dda31c1b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=489d21703ba940c1857622700a940ddc, user_id=05a3371f82b3480b9be0e0264585a678, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:02.783267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4da27d-a304-46a0-8f84-297dda31c1b4 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:02.797193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1212/2418] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:02 2022] PUT /identity/v3/projects/489d21703ba940c1857622700a940ddc/users/05a3371f82b3480b9be0e0264585a678/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:11:02.808218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59d81d-7187-4336-aee1-045a026a3481 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.808544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59d81d-7187-4336-aee1-045a026a3481 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.808826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59d81d-7187-4336-aee1-045a026a3481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.835271 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-bb59d81d-7187-4336-aee1-045a026a3481 None None] MFA Rules not processed for user `05a3371f82b3480b9be0e0264585a678`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:02.836466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-bb59d81d-7187-4336-aee1-045a026a3481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:02.906265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1207/2419] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:02.907865 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4da27d-a304-46a0-8f84-297dda31c1b4 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:02.923400 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:02.979421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4da27d-a304-46a0-8f84-297dda31c1b4 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:02.984346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08695-6962-4f2a-8736-a35dc2910cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:02.985109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08695-6962-4f2a-8736-a35dc2910cad admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:02.985516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08695-6962-4f2a-8736-a35dc2910cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:02.985671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b08695-6962-4f2a-8736-a35dc2910cad admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:02.986365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b08695-6962-4f2a-8736-a35dc2910cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:02.988284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b08695-6962-4f2a-8736-a35dc2910cad admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:03.015392 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1213/2420] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:11:02 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:03.019275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59d81d-7187-4336-aee1-045a026a3481 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:03.029773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59d81d-7187-4336-aee1-045a026a3481 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:03.086022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f0dce-c500-4e4b-afc4-39839afe1ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:03.086615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f0dce-c500-4e4b-afc4-39839afe1ffb admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:03.086901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f0dce-c500-4e4b-afc4-39839afe1ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:03.087176 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f0dce-c500-4e4b-afc4-39839afe1ffb admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:03.087864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f0dce-c500-4e4b-afc4-39839afe1ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:03.089833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f0dce-c500-4e4b-afc4-39839afe1ffb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:03.097239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:03.110206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1208/2421] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:11:03 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:11:03.114152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08695-6962-4f2a-8736-a35dc2910cad admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:03.125753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b08695-6962-4f2a-8736-a35dc2910cad admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:03.130402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1b565-29e0-42dd-8444-ce92ad3d4954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:03.131050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b565-29e0-42dd-8444-ce92ad3d4954 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:03.131332 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b565-29e0-42dd-8444-ce92ad3d4954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:03.131644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b565-29e0-42dd-8444-ce92ad3d4954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:03.132416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1b565-29e0-42dd-8444-ce92ad3d4954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:03.134010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1b565-29e0-42dd-8444-ce92ad3d4954 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:03.144605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1214/2422] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:03.149049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f0dce-c500-4e4b-afc4-39839afe1ffb admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:03.159002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f0dce-c500-4e4b-afc4-39839afe1ffb admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:03.164115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425512c1-6595-4c1c-9d35-f537d2dcc6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:03.165008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425512c1-6595-4c1c-9d35-f537d2dcc6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:03.165288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425512c1-6595-4c1c-9d35-f537d2dcc6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:03.165557 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425512c1-6595-4c1c-9d35-f537d2dcc6f2 admin admin] PATH_INFO: `/v3/projects/2f34653338934845bea5fc36df02b8b4/users/3d8bb09d349a4dabbf3bbaa65b30b6a7/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:03.180531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425512c1-6595-4c1c-9d35-f537d2dcc6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f34653338934845bea5fc36df02b8b4, user_id=3d8bb09d349a4dabbf3bbaa65b30b6a7, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:03.183163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425512c1-6595-4c1c-9d35-f537d2dcc6f2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:03.195966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1209/2423] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:03 2022] PUT /identity/v3/projects/2f34653338934845bea5fc36df02b8b4/users/3d8bb09d349a4dabbf3bbaa65b30b6a7/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:03.199837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1b565-29e0-42dd-8444-ce92ad3d4954 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:03.209746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1b565-29e0-42dd-8444-ce92ad3d4954 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:03.265593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316b2ad-8dad-4029-ac8e-acd9fe123548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:03.266253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316b2ad-8dad-4029-ac8e-acd9fe123548 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:03.266538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316b2ad-8dad-4029-ac8e-acd9fe123548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:03.266816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316b2ad-8dad-4029-ac8e-acd9fe123548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:03.267533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316b2ad-8dad-4029-ac8e-acd9fe123548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:03.269162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316b2ad-8dad-4029-ac8e-acd9fe123548 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:03.278836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1215/2424] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:03.283231 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425512c1-6595-4c1c-9d35-f537d2dcc6f2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:03.294026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425512c1-6595-4c1c-9d35-f537d2dcc6f2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:03.298674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f7e24-3b69-48e1-96b3-9e634c098abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:03.299555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f7e24-3b69-48e1-96b3-9e634c098abb admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:03.299842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f7e24-3b69-48e1-96b3-9e634c098abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:03.300120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f7e24-3b69-48e1-96b3-9e634c098abb admin admin] PATH_INFO: `/v3/projects/2f34653338934845bea5fc36df02b8b4/users/3d8bb09d349a4dabbf3bbaa65b30b6a7/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:03.302126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f7e24-3b69-48e1-96b3-9e634c098abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f34653338934845bea5fc36df02b8b4, user_id=3d8bb09d349a4dabbf3bbaa65b30b6a7, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:03.305527 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f7e24-3b69-48e1-96b3-9e634c098abb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:03.315642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1210/2425] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:03 2022] PUT /identity/v3/projects/2f34653338934845bea5fc36df02b8b4/users/3d8bb09d349a4dabbf3bbaa65b30b6a7/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:03.325517 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41f6c0-6d39-486b-b820-f0812035d4d2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:03.325839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41f6c0-6d39-486b-b820-f0812035d4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:03.326121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41f6c0-6d39-486b-b820-f0812035d4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:03.353064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ee41f6c0-6d39-486b-b820-f0812035d4d2 None None] MFA Rules not processed for user `3d8bb09d349a4dabbf3bbaa65b30b6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:03.354207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ee41f6c0-6d39-486b-b820-f0812035d4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:03.423149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1216/2426] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:03.434338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f7e24-3b69-48e1-96b3-9e634c098abb admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:03.436025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b36f7e24-3b69-48e1-96b3-9e634c098abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:03.492798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f7e24-3b69-48e1-96b3-9e634c098abb admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:03.496335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4f60b-39e5-4c5a-ba60-f57633db091c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:03.497061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4f60b-39e5-4c5a-ba60-f57633db091c service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:03.497341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4f60b-39e5-4c5a-ba60-f57633db091c service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:03.497613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4f60b-39e5-4c5a-ba60-f57633db091c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:03.506429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4f60b-39e5-4c5a-ba60-f57633db091c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:03.508181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4f60b-39e5-4c5a-ba60-f57633db091c service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:03.517462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1211/2427] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:03.915150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41f6c0-6d39-486b-b820-f0812035d4d2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:03.926049 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:03.979724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41f6c0-6d39-486b-b820-f0812035d4d2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:03.984258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:03.985162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:03.985439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:03.985718 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] PATH_INFO: `/v3/users/05a3371f82b3480b9be0e0264585a678` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:03.986896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] RBAC: Authorizing `identity:delete_user(user_id=05a3371f82b3480b9be0e0264585a678)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:03.988875 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:04.036373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05a3371f82b3480b9be0e0264585a678'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.037372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05a3371f82b3480b9be0e0264585a678'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.044888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05a3371f82b3480b9be0e0264585a678'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.045849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05a3371f82b3480b9be0e0264585a678'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.058284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05a3371f82b3480b9be0e0264585a678'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.071306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1217/2428] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:11:03 2022] DELETE /identity/v3/users/05a3371f82b3480b9be0e0264585a678 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:04.075920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44feaba-ba4c-401d-b3ac-7b39df14c38e None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:04.076460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44feaba-ba4c-401d-b3ac-7b39df14c38e None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:04.076586 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44feaba-ba4c-401d-b3ac-7b39df14c38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:04.094346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e44feaba-ba4c-401d-b3ac-7b39df14c38e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:04.095543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e44feaba-ba4c-401d-b3ac-7b39df14c38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:04.157045 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1212/2429] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:11:04.166941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:04.168479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:04.225036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb9de5-53d2-455d-8ee5-03edf616d25d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:04.229344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175481a2-68da-4176-b2a9-12b80b45de8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:04.229982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175481a2-68da-4176-b2a9-12b80b45de8e service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:04.230265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175481a2-68da-4176-b2a9-12b80b45de8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:04.230540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175481a2-68da-4176-b2a9-12b80b45de8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:04.239451 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175481a2-68da-4176-b2a9-12b80b45de8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:04.240975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175481a2-68da-4176-b2a9-12b80b45de8e service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:04.250343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1218/2430] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:04.523392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44feaba-ba4c-401d-b3ac-7b39df14c38e None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:04.524531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e44feaba-ba4c-401d-b3ac-7b39df14c38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:04.536604 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44feaba-ba4c-401d-b3ac-7b39df14c38e None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:04.542222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:04.543086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:04.543368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:04.543641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] PATH_INFO: `/v3/projects/489d21703ba940c1857622700a940ddc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:04.544845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] RBAC: Authorizing `identity:delete_project(project_id=489d21703ba940c1857622700a940ddc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:04.546768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:04.593654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '489d21703ba940c1857622700a940ddc', 'request_id': 'req-91777f3a-de6a-4002-bbfd-76a45d57562e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.595551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:04.600820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '489d21703ba940c1857622700a940ddc', 'request_id': 'req-91777f3a-de6a-4002-bbfd-76a45d57562e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.608613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '489d21703ba940c1857622700a940ddc', 'request_id': 'req-91777f3a-de6a-4002-bbfd-76a45d57562e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.619409 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] The token cache is being invalidate because project 489d21703ba940c1857622700a940ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:11:04.619699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.620383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.622036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1213/2431] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:11:04 2022] DELETE /identity/v3/projects/489d21703ba940c1857622700a940ddc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:04.626150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175481a2-68da-4176-b2a9-12b80b45de8e service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:04.636066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175481a2-68da-4176-b2a9-12b80b45de8e service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:04.665876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:04.695450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:04.696927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:04.697385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:04.697856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] PATH_INFO: `/v3/users/3d8bb09d349a4dabbf3bbaa65b30b6a7` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:04.699891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d8bb09d349a4dabbf3bbaa65b30b6a7)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:04.702405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:04.748052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d8bb09d349a4dabbf3bbaa65b30b6a7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.748713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d8bb09d349a4dabbf3bbaa65b30b6a7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.755654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d8bb09d349a4dabbf3bbaa65b30b6a7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.756355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d8bb09d349a4dabbf3bbaa65b30b6a7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.768365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-82fa7626-5008-4771-be49-e5fa208b84d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d8bb09d349a4dabbf3bbaa65b30b6a7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:04.781605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1219/2432] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:11:04 2022] DELETE /identity/v3/users/3d8bb09d349a4dabbf3bbaa65b30b6a7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:05.050043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.051488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:05.064550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91777f3a-de6a-4002-bbfd-76a45d57562e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.123010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.124204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.124513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.124787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] PATH_INFO: `/v3/projects/2f34653338934845bea5fc36df02b8b4` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.125996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f34653338934845bea5fc36df02b8b4)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:05.127901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:05.174290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f34653338934845bea5fc36df02b8b4', 'request_id': 'req-84634735-def8-4206-a799-e3aa47f1e194'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:05.180856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f34653338934845bea5fc36df02b8b4', 'request_id': 'req-84634735-def8-4206-a799-e3aa47f1e194'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:05.188527 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f34653338934845bea5fc36df02b8b4', 'request_id': 'req-84634735-def8-4206-a799-e3aa47f1e194'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:05.199122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] The token cache is being invalidate because project 2f34653338934845bea5fc36df02b8b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:11:05.199408 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:05.200044 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-84634735-def8-4206-a799-e3aa47f1e194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:05.202857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1214/2433] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:11:05 2022] DELETE /identity/v3/projects/2f34653338934845bea5fc36df02b8b4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:05.206620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a324f-98d3-457c-a1a0-3ff0dfd98ca2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.207100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a324f-98d3-457c-a1a0-3ff0dfd98ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.207197 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a324f-98d3-457c-a1a0-3ff0dfd98ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.226263 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-d21a324f-98d3-457c-a1a0-3ff0dfd98ca2 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:05.227458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d21a324f-98d3-457c-a1a0-3ff0dfd98ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:05.289568 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1220/2434] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:05.297428 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573127d-6d83-4893-9b49-0f599451512f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.297755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573127d-6d83-4893-9b49-0f599451512f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.298034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573127d-6d83-4893-9b49-0f599451512f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.317650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c573127d-6d83-4893-9b49-0f599451512f None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:05.318874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c573127d-6d83-4893-9b49-0f599451512f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:05.327642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1215/2435] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:05.331807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a324f-98d3-457c-a1a0-3ff0dfd98ca2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.341831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a324f-98d3-457c-a1a0-3ff0dfd98ca2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.346503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb75aa-5ed1-4665-8356-fb8e29226b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.347159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb75aa-5ed1-4665-8356-fb8e29226b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.347441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb75aa-5ed1-4665-8356-fb8e29226b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.347711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb75aa-5ed1-4665-8356-fb8e29226b02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.348484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb75aa-5ed1-4665-8356-fb8e29226b02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:11:05.348787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb75aa-5ed1-4665-8356-fb8e29226b02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:05.350941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb75aa-5ed1-4665-8356-fb8e29226b02 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:05.362555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1221/2436] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 02:11:05.367183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573127d-6d83-4893-9b49-0f599451512f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.376483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573127d-6d83-4893-9b49-0f599451512f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.381099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d167f9-119d-40ea-8b4c-d46f78844c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.382278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d167f9-119d-40ea-8b4c-d46f78844c63 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.382574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d167f9-119d-40ea-8b4c-d46f78844c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.382830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d167f9-119d-40ea-8b4c-d46f78844c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.383517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d167f9-119d-40ea-8b4c-d46f78844c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:05.385445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d167f9-119d-40ea-8b4c-d46f78844c63 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:05.411003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1216/2437] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:11:05 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:11:05.415159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb75aa-5ed1-4665-8356-fb8e29226b02 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.424984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb75aa-5ed1-4665-8356-fb8e29226b02 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.459110 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:05.483559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f2787-1373-4147-96d9-c73e18f3ee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.484259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f2787-1373-4147-96d9-c73e18f3ee9a admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.484590 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f2787-1373-4147-96d9-c73e18f3ee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.484855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f2787-1373-4147-96d9-c73e18f3ee9a admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.485546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f2787-1373-4147-96d9-c73e18f3ee9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:05.487494 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f2787-1373-4147-96d9-c73e18f3ee9a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:05.507948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1222/2438] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:11:05 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:11:05.511856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d167f9-119d-40ea-8b4c-d46f78844c63 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.522891 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d167f9-119d-40ea-8b4c-d46f78844c63 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.527807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850369c-82fe-4c20-9c7d-0e7426edd8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.528505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850369c-82fe-4c20-9c7d-0e7426edd8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.528790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850369c-82fe-4c20-9c7d-0e7426edd8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.529067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850369c-82fe-4c20-9c7d-0e7426edd8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.529789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850369c-82fe-4c20-9c7d-0e7426edd8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:05.531641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850369c-82fe-4c20-9c7d-0e7426edd8e5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:05.542550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1217/2439] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:05.546616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f2787-1373-4147-96d9-c73e18f3ee9a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.557097 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f2787-1373-4147-96d9-c73e18f3ee9a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.561711 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd73261-cfac-4bc9-a747-542155b4ba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.562364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd73261-cfac-4bc9-a747-542155b4ba9c admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.562651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd73261-cfac-4bc9-a747-542155b4ba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.562930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd73261-cfac-4bc9-a747-542155b4ba9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.563651 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd73261-cfac-4bc9-a747-542155b4ba9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:05.565288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd73261-cfac-4bc9-a747-542155b4ba9c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:05.575834 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1223/2440] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:05.579850 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850369c-82fe-4c20-9c7d-0e7426edd8e5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.590553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850369c-82fe-4c20-9c7d-0e7426edd8e5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.595147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91727c2-0e61-448b-bad0-ff97fdc2fca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.596157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91727c2-0e61-448b-bad0-ff97fdc2fca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.596473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91727c2-0e61-448b-bad0-ff97fdc2fca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.596755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91727c2-0e61-448b-bad0-ff97fdc2fca8 admin admin] PATH_INFO: `/v3/projects/861e561fa6ce4ee79b958c19b57aaf00/users/03d3b7f4134a43ec85995c262dcaaa36/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.611249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91727c2-0e61-448b-bad0-ff97fdc2fca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=861e561fa6ce4ee79b958c19b57aaf00, user_id=03d3b7f4134a43ec85995c262dcaaa36, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:05.613922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91727c2-0e61-448b-bad0-ff97fdc2fca8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:05.626435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1218/2441] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:05 2022] PUT /identity/v3/projects/861e561fa6ce4ee79b958c19b57aaf00/users/03d3b7f4134a43ec85995c262dcaaa36/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:05.638410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115b8c9-8d9f-4847-b86d-5795e1efecd1 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.638707 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115b8c9-8d9f-4847-b86d-5795e1efecd1 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.638988 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3115b8c9-8d9f-4847-b86d-5795e1efecd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.665557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3115b8c9-8d9f-4847-b86d-5795e1efecd1 None None] MFA Rules not processed for user `03d3b7f4134a43ec85995c262dcaaa36`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:05.666742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3115b8c9-8d9f-4847-b86d-5795e1efecd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:05.731203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1224/2442] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:05.735728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91727c2-0e61-448b-bad0-ff97fdc2fca8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.745579 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:05.798996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91727c2-0e61-448b-bad0-ff97fdc2fca8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.804275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc4bfc-60ac-4718-8b51-9dd383c51f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.804956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc4bfc-60ac-4718-8b51-9dd383c51f13 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.805240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc4bfc-60ac-4718-8b51-9dd383c51f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.805514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc4bfc-60ac-4718-8b51-9dd383c51f13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.806196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc4bfc-60ac-4718-8b51-9dd383c51f13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:05.808103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc4bfc-60ac-4718-8b51-9dd383c51f13 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:05.833339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1219/2443] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:11:05 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:05.837558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115b8c9-8d9f-4847-b86d-5795e1efecd1 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.848225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3115b8c9-8d9f-4847-b86d-5795e1efecd1 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.905690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b82db-5dcb-4f0c-957a-136c1a0f4bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.906609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b82db-5dcb-4f0c-957a-136c1a0f4bfd admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.906899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b82db-5dcb-4f0c-957a-136c1a0f4bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.907176 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b82db-5dcb-4f0c-957a-136c1a0f4bfd admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.907864 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b82db-5dcb-4f0c-957a-136c1a0f4bfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:05.909866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b82db-5dcb-4f0c-957a-136c1a0f4bfd admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:05.933142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1225/2444] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:11:05 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:05.937333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc4bfc-60ac-4718-8b51-9dd383c51f13 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.948607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc4bfc-60ac-4718-8b51-9dd383c51f13 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.953399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa35fd-2a33-43d1-93a5-c96ae3f19a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.954090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa35fd-2a33-43d1-93a5-c96ae3f19a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.954375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa35fd-2a33-43d1-93a5-c96ae3f19a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.954651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa35fd-2a33-43d1-93a5-c96ae3f19a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:05.955397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa35fd-2a33-43d1-93a5-c96ae3f19a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:05.957076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa35fd-2a33-43d1-93a5-c96ae3f19a79 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:05.968203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1220/2445] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:05.972323 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b82db-5dcb-4f0c-957a-136c1a0f4bfd admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:05.982904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b82db-5dcb-4f0c-957a-136c1a0f4bfd admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:05.987438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11677db-b071-45bf-96c3-b64b5a1a5103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:05.988335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11677db-b071-45bf-96c3-b64b5a1a5103 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:05.988654 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11677db-b071-45bf-96c3-b64b5a1a5103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:05.988930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11677db-b071-45bf-96c3-b64b5a1a5103 admin admin] PATH_INFO: `/v3/projects/6b2757c6dc364188bff8df69d5f54596/users/a14e6ac994174717b9ce860184351ac7/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:06.004236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11677db-b071-45bf-96c3-b64b5a1a5103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2757c6dc364188bff8df69d5f54596, user_id=a14e6ac994174717b9ce860184351ac7, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:06.008044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11677db-b071-45bf-96c3-b64b5a1a5103 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:06.021243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1226/2446] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:05 2022] PUT /identity/v3/projects/6b2757c6dc364188bff8df69d5f54596/users/a14e6ac994174717b9ce860184351ac7/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:06.025329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa35fd-2a33-43d1-93a5-c96ae3f19a79 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:06.035544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa35fd-2a33-43d1-93a5-c96ae3f19a79 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:06.091720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b5b72-e4aa-48ac-a332-b8eedc93b3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:06.092389 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b5b72-e4aa-48ac-a332-b8eedc93b3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:06.092700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b5b72-e4aa-48ac-a332-b8eedc93b3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:06.092977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b5b72-e4aa-48ac-a332-b8eedc93b3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:06.093697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b5b72-e4aa-48ac-a332-b8eedc93b3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:06.095302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b5b72-e4aa-48ac-a332-b8eedc93b3a4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:06.105872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1221/2447] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:06.109870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11677db-b071-45bf-96c3-b64b5a1a5103 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:06.120759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11677db-b071-45bf-96c3-b64b5a1a5103 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:06.125339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a799e11-2dac-454d-8356-dfa735944896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:06.126211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a799e11-2dac-454d-8356-dfa735944896 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:06.126497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a799e11-2dac-454d-8356-dfa735944896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:06.126776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a799e11-2dac-454d-8356-dfa735944896 admin admin] PATH_INFO: `/v3/projects/6b2757c6dc364188bff8df69d5f54596/users/a14e6ac994174717b9ce860184351ac7/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:06.128766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a799e11-2dac-454d-8356-dfa735944896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2757c6dc364188bff8df69d5f54596, user_id=a14e6ac994174717b9ce860184351ac7, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:06.131451 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a799e11-2dac-454d-8356-dfa735944896 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:06.137023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:06.142847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1227/2448] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:06 2022] PUT /identity/v3/projects/6b2757c6dc364188bff8df69d5f54596/users/a14e6ac994174717b9ce860184351ac7/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:06.153126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e38ed-d3ab-4629-85d0-d93836045548 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:06.153621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e38ed-d3ab-4629-85d0-d93836045548 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:06.153739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e38ed-d3ab-4629-85d0-d93836045548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:06.180353 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-725e38ed-d3ab-4629-85d0-d93836045548 None None] MFA Rules not processed for user `a14e6ac994174717b9ce860184351ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:06.181609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-725e38ed-d3ab-4629-85d0-d93836045548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:06.248782 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1222/2449] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:06.260928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a799e11-2dac-454d-8356-dfa735944896 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:06.262637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2a799e11-2dac-454d-8356-dfa735944896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:06.320555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a799e11-2dac-454d-8356-dfa735944896 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:06.324198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c977abf-8bab-4450-8d10-c3e7ede8d6dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:06.324866 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c977abf-8bab-4450-8d10-c3e7ede8d6dc service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:06.325156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c977abf-8bab-4450-8d10-c3e7ede8d6dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:06.325435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c977abf-8bab-4450-8d10-c3e7ede8d6dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:06.334300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c977abf-8bab-4450-8d10-c3e7ede8d6dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:06.336305 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c977abf-8bab-4450-8d10-c3e7ede8d6dc service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:06.345573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1228/2450] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:06.435017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e38ed-d3ab-4629-85d0-d93836045548 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:06.439930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:06.447612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e38ed-d3ab-4629-85d0-d93836045548 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:06.451326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbd600-6e2a-437d-a098-28da88695cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:06.451973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbd600-6e2a-437d-a098-28da88695cad service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:06.452258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbd600-6e2a-437d-a098-28da88695cad service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:06.452574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbd600-6e2a-437d-a098-28da88695cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:06.511975 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbd600-6e2a-437d-a098-28da88695cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:06.513664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbd600-6e2a-437d-a098-28da88695cad service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:06.523958 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1223/2451] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:06.777360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c977abf-8bab-4450-8d10-c3e7ede8d6dc service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:06.842601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c977abf-8bab-4450-8d10-c3e7ede8d6dc service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:06.847159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:06.848025 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:06.848313 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:06.848613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] PATH_INFO: `/v3/users/03d3b7f4134a43ec85995c262dcaaa36` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:06.849792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d3b7f4134a43ec85995c262dcaaa36)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:06.852565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:06.898278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d3b7f4134a43ec85995c262dcaaa36'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:06.898925 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d3b7f4134a43ec85995c262dcaaa36'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:06.905705 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d3b7f4134a43ec85995c262dcaaa36'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:06.906565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d3b7f4134a43ec85995c262dcaaa36'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:06.919935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d3b7f4134a43ec85995c262dcaaa36'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:06.932765 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1229/2452] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:11:06 2022] DELETE /identity/v3/users/03d3b7f4134a43ec85995c262dcaaa36 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:06.937178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aeaa32-9440-47c2-96ef-9e059d65cadd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:06.937603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aeaa32-9440-47c2-96ef-9e059d65cadd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:06.937916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aeaa32-9440-47c2-96ef-9e059d65cadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:06.955403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f6aeaa32-9440-47c2-96ef-9e059d65cadd None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:06.956628 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f6aeaa32-9440-47c2-96ef-9e059d65cadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:07.015330 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1224/2453] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:07.024800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:07.026197 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:07.048635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:07.084906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120f483-df60-4707-b49d-3d6c6e471d43 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:07.088545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06e855-999e-4f89-9a1c-f687873b816b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:07.089183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06e855-999e-4f89-9a1c-f687873b816b service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:07.089465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06e855-999e-4f89-9a1c-f687873b816b service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:07.089741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06e855-999e-4f89-9a1c-f687873b816b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:07.098728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06e855-999e-4f89-9a1c-f687873b816b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:07.100687 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06e855-999e-4f89-9a1c-f687873b816b service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:07.110064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1230/2454] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:07.371484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aeaa32-9440-47c2-96ef-9e059d65cadd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:07.372994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f6aeaa32-9440-47c2-96ef-9e059d65cadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:07.386323 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aeaa32-9440-47c2-96ef-9e059d65cadd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:07.391236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:07.392147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:07.392458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:07.392741 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] PATH_INFO: `/v3/projects/861e561fa6ce4ee79b958c19b57aaf00` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:07.393980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=861e561fa6ce4ee79b958c19b57aaf00)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:07.395910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:07.445258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861e561fa6ce4ee79b958c19b57aaf00', 'request_id': 'req-8b41f48f-b7df-4e8b-9510-42f9328a64f6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:07.451278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861e561fa6ce4ee79b958c19b57aaf00', 'request_id': 'req-8b41f48f-b7df-4e8b-9510-42f9328a64f6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:07.457913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861e561fa6ce4ee79b958c19b57aaf00', 'request_id': 'req-8b41f48f-b7df-4e8b-9510-42f9328a64f6'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:07.468571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] The token cache is being invalidate because project 861e561fa6ce4ee79b958c19b57aaf00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:11:07.468861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:07.469479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:07.471010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1225/2455] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:11:07 2022] DELETE /identity/v3/projects/861e561fa6ce4ee79b958c19b57aaf00 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:07.475001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06e855-999e-4f89-9a1c-f687873b816b service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:07.484978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06e855-999e-4f89-9a1c-f687873b816b service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:07.543811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:07.544812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:07.545045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:07.545326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] PATH_INFO: `/v3/users/a14e6ac994174717b9ce860184351ac7` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:07.546521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a14e6ac994174717b9ce860184351ac7)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:07.548483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:07.597204 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a14e6ac994174717b9ce860184351ac7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:07.598304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a14e6ac994174717b9ce860184351ac7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:07.605973 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a14e6ac994174717b9ce860184351ac7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:07.606612 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a14e6ac994174717b9ce860184351ac7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:07.619215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-3dbe941d-1436-4a67-a373-f7aea29f128c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a14e6ac994174717b9ce860184351ac7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:07.634727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1231/2456] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:11:07 2022] DELETE /identity/v3/users/a14e6ac994174717b9ce860184351ac7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:08.152086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:08.153558 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:08.169115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41f48f-b7df-4e8b-9510-42f9328a64f6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:08.195635 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:08.242500 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:08.243366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.243642 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.243908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] PATH_INFO: `/v3/projects/6b2757c6dc364188bff8df69d5f54596` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.245088 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b2757c6dc364188bff8df69d5f54596)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:08.247061 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:08.295263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b2757c6dc364188bff8df69d5f54596', 'request_id': 'req-23c94a44-6f46-43bd-8c0a-7def3ad292b5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:08.301474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b2757c6dc364188bff8df69d5f54596', 'request_id': 'req-23c94a44-6f46-43bd-8c0a-7def3ad292b5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:08.308704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b2757c6dc364188bff8df69d5f54596', 'request_id': 'req-23c94a44-6f46-43bd-8c0a-7def3ad292b5'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:08.320001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] The token cache is being invalidate because project 6b2757c6dc364188bff8df69d5f54596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:11:08.320288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:08.320942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-23c94a44-6f46-43bd-8c0a-7def3ad292b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:08.322521 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1226/2457] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:11:08 2022] DELETE /identity/v3/projects/6b2757c6dc364188bff8df69d5f54596 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:08.327418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459cdb8-94cc-4cd7-8fe3-ba63112eda96 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.327723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459cdb8-94cc-4cd7-8fe3-ba63112eda96 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.328022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459cdb8-94cc-4cd7-8fe3-ba63112eda96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.345764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-d459cdb8-94cc-4cd7-8fe3-ba63112eda96 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:08.346950 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d459cdb8-94cc-4cd7-8fe3-ba63112eda96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:08.384425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:08.407346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1232/2458] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:11:08.416234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203e86e-3302-41a4-89ed-8b3a21427db8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.416743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203e86e-3302-41a4-89ed-8b3a21427db8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.416837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203e86e-3302-41a4-89ed-8b3a21427db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.435691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6203e86e-3302-41a4-89ed-8b3a21427db8 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:08.436944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6203e86e-3302-41a4-89ed-8b3a21427db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:08.445293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1227/2459] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:08.449445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459cdb8-94cc-4cd7-8fe3-ba63112eda96 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:08.458938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459cdb8-94cc-4cd7-8fe3-ba63112eda96 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:08.464102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443561f1-8c1a-4459-8ddf-daaf205ad581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:08.464784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443561f1-8c1a-4459-8ddf-daaf205ad581 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.465066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443561f1-8c1a-4459-8ddf-daaf205ad581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.465343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443561f1-8c1a-4459-8ddf-daaf205ad581 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.466063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443561f1-8c1a-4459-8ddf-daaf205ad581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:11:08.466369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443561f1-8c1a-4459-8ddf-daaf205ad581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:08.467962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443561f1-8c1a-4459-8ddf-daaf205ad581 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:08.479423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1233/2460] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:11:08.483641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203e86e-3302-41a4-89ed-8b3a21427db8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:08.492978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203e86e-3302-41a4-89ed-8b3a21427db8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:08.497615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8ef0d-1311-4b38-a765-41c21801b53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:08.498266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8ef0d-1311-4b38-a765-41c21801b53d admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.498550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8ef0d-1311-4b38-a765-41c21801b53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.498827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8ef0d-1311-4b38-a765-41c21801b53d admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.499506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8ef0d-1311-4b38-a765-41c21801b53d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:08.501476 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8ef0d-1311-4b38-a765-41c21801b53d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:08.527238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1228/2461] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:11:08 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:08.531218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443561f1-8c1a-4459-8ddf-daaf205ad581 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:08.541580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443561f1-8c1a-4459-8ddf-daaf205ad581 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:08.597511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7b46-86c0-4ea3-a424-421bc8a31011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:08.598173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7b46-86c0-4ea3-a424-421bc8a31011 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.598455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7b46-86c0-4ea3-a424-421bc8a31011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.598734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7b46-86c0-4ea3-a424-421bc8a31011 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.599411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c7b46-86c0-4ea3-a424-421bc8a31011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:08.601461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c7b46-86c0-4ea3-a424-421bc8a31011 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:08.621516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1234/2462] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:11:08 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:08.625461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8ef0d-1311-4b38-a765-41c21801b53d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:08.636286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8ef0d-1311-4b38-a765-41c21801b53d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:08.641012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e4f74-4bff-4f4e-8ae7-2aa8ca92bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:08.641666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e4f74-4bff-4f4e-8ae7-2aa8ca92bc51 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.642041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e4f74-4bff-4f4e-8ae7-2aa8ca92bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.642327 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e4f74-4bff-4f4e-8ae7-2aa8ca92bc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.643037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e4f74-4bff-4f4e-8ae7-2aa8ca92bc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:08.644649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e4f74-4bff-4f4e-8ae7-2aa8ca92bc51 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:08.655477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1229/2463] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:08.659558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7b46-86c0-4ea3-a424-421bc8a31011 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:08.670018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7b46-86c0-4ea3-a424-421bc8a31011 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:08.674744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6270fb-b5a5-4654-90dc-b9c51ab55c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:08.675387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6270fb-b5a5-4654-90dc-b9c51ab55c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.675670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6270fb-b5a5-4654-90dc-b9c51ab55c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.675947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6270fb-b5a5-4654-90dc-b9c51ab55c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.676695 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6270fb-b5a5-4654-90dc-b9c51ab55c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:08.679523 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6270fb-b5a5-4654-90dc-b9c51ab55c17 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:08.689404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1235/2464] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:08.693563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e4f74-4bff-4f4e-8ae7-2aa8ca92bc51 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:08.704986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e4f74-4bff-4f4e-8ae7-2aa8ca92bc51 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:08.709745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ec9ed-ba5f-42c4-85dd-5cc08b0ed406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:08.710714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ec9ed-ba5f-42c4-85dd-5cc08b0ed406 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.710992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ec9ed-ba5f-42c4-85dd-5cc08b0ed406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.711266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ec9ed-ba5f-42c4-85dd-5cc08b0ed406 admin admin] PATH_INFO: `/v3/projects/9037b0248e6f4b02a4c278d3bb437c3f/users/65f370c0b64b4e1aaec177ce4e60a148/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.726151 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ec9ed-ba5f-42c4-85dd-5cc08b0ed406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9037b0248e6f4b02a4c278d3bb437c3f, user_id=65f370c0b64b4e1aaec177ce4e60a148, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:08.728836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ec9ed-ba5f-42c4-85dd-5cc08b0ed406 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:08.744674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1230/2465] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:08 2022] PUT /identity/v3/projects/9037b0248e6f4b02a4c278d3bb437c3f/users/65f370c0b64b4e1aaec177ce4e60a148/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:08.754991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac257f-26d7-4b54-a99c-f805da8b5b88 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.755280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac257f-26d7-4b54-a99c-f805da8b5b88 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.755566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac257f-26d7-4b54-a99c-f805da8b5b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.783742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7cac257f-26d7-4b54-a99c-f805da8b5b88 None None] MFA Rules not processed for user `65f370c0b64b4e1aaec177ce4e60a148`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:08.785056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7cac257f-26d7-4b54-a99c-f805da8b5b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:08.855219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1236/2466] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:11:08.869212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ec9ed-ba5f-42c4-85dd-5cc08b0ed406 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:08.871068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-753ec9ed-ba5f-42c4-85dd-5cc08b0ed406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:08.930427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ec9ed-ba5f-42c4-85dd-5cc08b0ed406 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:08.934089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c8da1-317a-4efe-876c-05b44238ef50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:08.934741 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8da1-317a-4efe-876c-05b44238ef50 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:08.935019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8da1-317a-4efe-876c-05b44238ef50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:08.935291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c8da1-317a-4efe-876c-05b44238ef50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:08.940458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:08.945955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c8da1-317a-4efe-876c-05b44238ef50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:08.948151 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c8da1-317a-4efe-876c-05b44238ef50 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:08.957116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1231/2467] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:16.326159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac257f-26d7-4b54-a99c-f805da8b5b88 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:16.370861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:16.398385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac257f-26d7-4b54-a99c-f805da8b5b88 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:16.402965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:16.403839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:16.404127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:16.404435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] PATH_INFO: `/v3/users/65f370c0b64b4e1aaec177ce4e60a148` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:16.405646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f370c0b64b4e1aaec177ce4e60a148)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:16.407842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:16.454185 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f370c0b64b4e1aaec177ce4e60a148'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:16.454863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f370c0b64b4e1aaec177ce4e60a148'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:16.461880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f370c0b64b4e1aaec177ce4e60a148'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:16.462514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f370c0b64b4e1aaec177ce4e60a148'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:16.474431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f370c0b64b4e1aaec177ce4e60a148'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:16.487177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1237/2468] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:11:16 2022] DELETE /identity/v3/users/65f370c0b64b4e1aaec177ce4e60a148 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:16.492247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812cfdc-0944-498e-8663-d96b1d3c3fe1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:16.492804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812cfdc-0944-498e-8663-d96b1d3c3fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:16.492946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812cfdc-0944-498e-8663-d96b1d3c3fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:16.509522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1812cfdc-0944-498e-8663-d96b1d3c3fe1 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:16.510729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1812cfdc-0944-498e-8663-d96b1d3c3fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:16.571577 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1232/2469] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:16.581362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:16.582806 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:16.646551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8600ee-3e3d-4878-9eb0-8da9038b00a6 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:16.651528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca96ac-4050-4a65-946f-d7c1a819e4ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:16.652200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca96ac-4050-4a65-946f-d7c1a819e4ca service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:16.652655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca96ac-4050-4a65-946f-d7c1a819e4ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:16.652942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca96ac-4050-4a65-946f-d7c1a819e4ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:16.663825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca96ac-4050-4a65-946f-d7c1a819e4ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:16.665326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca96ac-4050-4a65-946f-d7c1a819e4ca service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:16.675969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1238/2470] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:16.956827 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812cfdc-0944-498e-8663-d96b1d3c3fe1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:16.958416 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1812cfdc-0944-498e-8663-d96b1d3c3fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:16.971707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812cfdc-0944-498e-8663-d96b1d3c3fe1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:16.977194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:16.978035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:16.978372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:16.978715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] PATH_INFO: `/v3/projects/9037b0248e6f4b02a4c278d3bb437c3f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:16.980002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9037b0248e6f4b02a4c278d3bb437c3f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:16.982035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:17.030207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9037b0248e6f4b02a4c278d3bb437c3f', 'request_id': 'req-0672bb0e-2714-4388-9ef4-76acad25d251'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:17.036774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9037b0248e6f4b02a4c278d3bb437c3f', 'request_id': 'req-0672bb0e-2714-4388-9ef4-76acad25d251'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:17.046377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9037b0248e6f4b02a4c278d3bb437c3f', 'request_id': 'req-0672bb0e-2714-4388-9ef4-76acad25d251'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:17.059050 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] The token cache is being invalidate because project 9037b0248e6f4b02a4c278d3bb437c3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:11:17.059348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:17.060187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0672bb0e-2714-4388-9ef4-76acad25d251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:17.062060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1233/2471] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:11:16 2022] DELETE /identity/v3/projects/9037b0248e6f4b02a4c278d3bb437c3f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:17.067372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00f004-9eaa-4d34-9f19-c4c77824062c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.067616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00f004-9eaa-4d34-9f19-c4c77824062c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.067922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00f004-9eaa-4d34-9f19-c4c77824062c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.086835 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ff00f004-9eaa-4d34-9f19-c4c77824062c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:17.088062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ff00f004-9eaa-4d34-9f19-c4c77824062c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:17.143231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:17.153511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1239/2472] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:11:17.161386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c478bc-a106-4037-9eeb-0bd6bf6c39fd None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.161675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c478bc-a106-4037-9eeb-0bd6bf6c39fd None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.161956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c478bc-a106-4037-9eeb-0bd6bf6c39fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.181294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a5c478bc-a106-4037-9eeb-0bd6bf6c39fd None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:17.182492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a5c478bc-a106-4037-9eeb-0bd6bf6c39fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:17.190418 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1234/2473] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:17.195061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00f004-9eaa-4d34-9f19-c4c77824062c None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:17.207017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00f004-9eaa-4d34-9f19-c4c77824062c None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:17.212175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1bd62-1153-4380-b31a-7346ea891d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:17.212880 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1bd62-1153-4380-b31a-7346ea891d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.213167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1bd62-1153-4380-b31a-7346ea891d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.213444 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1bd62-1153-4380-b31a-7346ea891d24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.214182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1bd62-1153-4380-b31a-7346ea891d24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:11:17.214493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1bd62-1153-4380-b31a-7346ea891d24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:17.216115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1bd62-1153-4380-b31a-7346ea891d24 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:17.231261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1240/2474] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:11:17.235279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c478bc-a106-4037-9eeb-0bd6bf6c39fd None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:17.245351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c478bc-a106-4037-9eeb-0bd6bf6c39fd None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:17.250029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c2381-a861-4b38-b65d-926be22f7d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:17.250693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c2381-a861-4b38-b65d-926be22f7d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.250972 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c2381-a861-4b38-b65d-926be22f7d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.251374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c2381-a861-4b38-b65d-926be22f7d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.252073 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c2381-a861-4b38-b65d-926be22f7d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:17.254187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c2381-a861-4b38-b65d-926be22f7d77 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:17.286288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1235/2475] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:11:17 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:17.291540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1bd62-1153-4380-b31a-7346ea891d24 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:17.302589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1bd62-1153-4380-b31a-7346ea891d24 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:17.370307 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e5818-6133-4a7a-ad6b-c845eae0f359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:17.371012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e5818-6133-4a7a-ad6b-c845eae0f359 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.371292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e5818-6133-4a7a-ad6b-c845eae0f359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.371576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e5818-6133-4a7a-ad6b-c845eae0f359 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.372387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e5818-6133-4a7a-ad6b-c845eae0f359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:17.374461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e5818-6133-4a7a-ad6b-c845eae0f359 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:17.412782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1241/2476] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:11:17 2022] POST /identity/v3/users => generated 281 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:11:17.415832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c2381-a861-4b38-b65d-926be22f7d77 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:17.435709 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c2381-a861-4b38-b65d-926be22f7d77 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:17.442633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559377-a791-4e69-8cc9-ae589196b255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:17.443666 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559377-a791-4e69-8cc9-ae589196b255 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.444237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559377-a791-4e69-8cc9-ae589196b255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.444929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559377-a791-4e69-8cc9-ae589196b255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.446715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559377-a791-4e69-8cc9-ae589196b255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:17.447943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559377-a791-4e69-8cc9-ae589196b255 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:17.466335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1236/2477] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:17.470483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e5818-6133-4a7a-ad6b-c845eae0f359 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:17.482108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e5818-6133-4a7a-ad6b-c845eae0f359 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:17.486801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8064ba-e620-488f-8ed2-a3d4fe45e2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:17.487457 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8064ba-e620-488f-8ed2-a3d4fe45e2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.487744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8064ba-e620-488f-8ed2-a3d4fe45e2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.488479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8064ba-e620-488f-8ed2-a3d4fe45e2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.489621 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8064ba-e620-488f-8ed2-a3d4fe45e2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:17.491953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8064ba-e620-488f-8ed2-a3d4fe45e2b7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:17.506992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1242/2478] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:11:17.508277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559377-a791-4e69-8cc9-ae589196b255 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:17.518487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559377-a791-4e69-8cc9-ae589196b255 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:17.523787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61ae53-af30-429d-a2ce-5e49abe60247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:17.524703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61ae53-af30-429d-a2ce-5e49abe60247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.524991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61ae53-af30-429d-a2ce-5e49abe60247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.525270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61ae53-af30-429d-a2ce-5e49abe60247 admin admin] PATH_INFO: `/v3/projects/8d1a113dcf204d139f61f0591a7e79fb/users/6de3f04b60cd431897bc824b12eaf393/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.541798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61ae53-af30-429d-a2ce-5e49abe60247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1a113dcf204d139f61f0591a7e79fb, user_id=6de3f04b60cd431897bc824b12eaf393, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:17.544486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61ae53-af30-429d-a2ce-5e49abe60247 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:17.558466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1237/2479] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:17 2022] PUT /identity/v3/projects/8d1a113dcf204d139f61f0591a7e79fb/users/6de3f04b60cd431897bc824b12eaf393/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:17.569689 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd250e-6c83-4f6e-ac73-8ac7e75eff82 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.570068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd250e-6c83-4f6e-ac73-8ac7e75eff82 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.570363 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd250e-6c83-4f6e-ac73-8ac7e75eff82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.597436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-dcbd250e-6c83-4f6e-ac73-8ac7e75eff82 None None] MFA Rules not processed for user `6de3f04b60cd431897bc824b12eaf393`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:17.598683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-dcbd250e-6c83-4f6e-ac73-8ac7e75eff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:17.667495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1243/2480] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:11:17.704022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61ae53-af30-429d-a2ce-5e49abe60247 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:17.705840 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3a61ae53-af30-429d-a2ce-5e49abe60247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:17.720591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:17.778230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61ae53-af30-429d-a2ce-5e49abe60247 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:17.784100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243580b5-5eef-4874-8178-5154fc9b197d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:17.785140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243580b5-5eef-4874-8178-5154fc9b197d service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.785640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243580b5-5eef-4874-8178-5154fc9b197d service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.786099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243580b5-5eef-4874-8178-5154fc9b197d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.799230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243580b5-5eef-4874-8178-5154fc9b197d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:17.801030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243580b5-5eef-4874-8178-5154fc9b197d service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:17.811957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1238/2481] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:17.827567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd250e-6c83-4f6e-ac73-8ac7e75eff82 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:17.892428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:17.898917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd250e-6c83-4f6e-ac73-8ac7e75eff82 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:17.905084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:17.906477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:17.908809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:17.909100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] PATH_INFO: `/v3/users/6de3f04b60cd431897bc824b12eaf393` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:17.910418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de3f04b60cd431897bc824b12eaf393)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:17.912360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:17.964171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de3f04b60cd431897bc824b12eaf393'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:17.965217 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de3f04b60cd431897bc824b12eaf393'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:17.972523 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de3f04b60cd431897bc824b12eaf393'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:17.973181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de3f04b60cd431897bc824b12eaf393'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:17.985743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de3f04b60cd431897bc824b12eaf393'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:17.998143 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1244/2482] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:11:17 2022] DELETE /identity/v3/users/6de3f04b60cd431897bc824b12eaf393 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:18.002789 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196102-3546-4cf2-acc5-dd7c7613f577 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.003084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196102-3546-4cf2-acc5-dd7c7613f577 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.003367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4196102-3546-4cf2-acc5-dd7c7613f577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.020873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f4196102-3546-4cf2-acc5-dd7c7613f577 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:18.022102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f4196102-3546-4cf2-acc5-dd7c7613f577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:18.080802 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1239/2483] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:18.090683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:18.092127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:18.154258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a3e99-d35d-4bb4-92a8-f6e0eb9b72da admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:18.158188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e95705-7cf0-49cb-a4d0-5eb35109879a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:18.158838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e95705-7cf0-49cb-a4d0-5eb35109879a service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.159127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e95705-7cf0-49cb-a4d0-5eb35109879a service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.159409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e95705-7cf0-49cb-a4d0-5eb35109879a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.169633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e95705-7cf0-49cb-a4d0-5eb35109879a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:18.171095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e95705-7cf0-49cb-a4d0-5eb35109879a service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:18.180913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1245/2484] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:18.461695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4196102-3546-4cf2-acc5-dd7c7613f577 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:18.463397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f4196102-3546-4cf2-acc5-dd7c7613f577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:18.476241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4196102-3546-4cf2-acc5-dd7c7613f577 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:18.481055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:18.482034 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.482437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.482809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] PATH_INFO: `/v3/projects/8d1a113dcf204d139f61f0591a7e79fb` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.484178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1a113dcf204d139f61f0591a7e79fb)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:18.486243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:18.539643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1a113dcf204d139f61f0591a7e79fb', 'request_id': 'req-9e8c159a-7ffe-4b21-84dc-021f525462ba'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:18.545750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1a113dcf204d139f61f0591a7e79fb', 'request_id': 'req-9e8c159a-7ffe-4b21-84dc-021f525462ba'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:18.554517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1a113dcf204d139f61f0591a7e79fb', 'request_id': 'req-9e8c159a-7ffe-4b21-84dc-021f525462ba'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:18.565679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] The token cache is being invalidate because project 8d1a113dcf204d139f61f0591a7e79fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:11:18.566022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:18.566654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-9e8c159a-7ffe-4b21-84dc-021f525462ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:18.568234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1240/2485] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:11:18 2022] DELETE /identity/v3/projects/8d1a113dcf204d139f61f0591a7e79fb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:18.573555 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63cfa2-809b-4939-b043-7851d4c0674a None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.574031 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63cfa2-809b-4939-b043-7851d4c0674a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.574130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63cfa2-809b-4939-b043-7851d4c0674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.592331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-1c63cfa2-809b-4939-b043-7851d4c0674a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:18.593559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1c63cfa2-809b-4939-b043-7851d4c0674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:18.654165 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1246/2486] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:11:18.661968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a45d54-951e-48b4-b9c0-c52c4fadd900 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.662260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a45d54-951e-48b4-b9c0-c52c4fadd900 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.662539 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a45d54-951e-48b4-b9c0-c52c4fadd900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.681212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-17a45d54-951e-48b4-b9c0-c52c4fadd900 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:18.682402 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-17a45d54-951e-48b4-b9c0-c52c4fadd900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:18.690194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1241/2487] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:18.694278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63cfa2-809b-4939-b043-7851d4c0674a None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:18.704859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63cfa2-809b-4939-b043-7851d4c0674a None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:18.709408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6f485-ef83-4b31-a67d-94926b55f8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:18.710433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6f485-ef83-4b31-a67d-94926b55f8cc admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.710721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6f485-ef83-4b31-a67d-94926b55f8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.711033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6f485-ef83-4b31-a67d-94926b55f8cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.711761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6f485-ef83-4b31-a67d-94926b55f8cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:11:18.712065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6f485-ef83-4b31-a67d-94926b55f8cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:18.713713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6f485-ef83-4b31-a67d-94926b55f8cc admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:18.725411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1247/2488] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:11:18.730450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a45d54-951e-48b4-b9c0-c52c4fadd900 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:18.740068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a45d54-951e-48b4-b9c0-c52c4fadd900 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:18.745856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1c7af-e1dd-41d7-bccf-1b8a1bdb0dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:18.746517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c7af-e1dd-41d7-bccf-1b8a1bdb0dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.746797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c7af-e1dd-41d7-bccf-1b8a1bdb0dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.747066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1c7af-e1dd-41d7-bccf-1b8a1bdb0dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.747762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1c7af-e1dd-41d7-bccf-1b8a1bdb0dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:18.749703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1c7af-e1dd-41d7-bccf-1b8a1bdb0dd0 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:18.780048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1242/2489] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:11:18 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:11:18.781604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6f485-ef83-4b31-a67d-94926b55f8cc admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:18.788601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:18.792607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6f485-ef83-4b31-a67d-94926b55f8cc admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:18.852443 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6f1be-fd72-4b27-b44d-11b4ee4a2758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:18.853122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f1be-fd72-4b27-b44d-11b4ee4a2758 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.853406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f1be-fd72-4b27-b44d-11b4ee4a2758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.853683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6f1be-fd72-4b27-b44d-11b4ee4a2758 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.854366 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6f1be-fd72-4b27-b44d-11b4ee4a2758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:18.856299 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6f1be-fd72-4b27-b44d-11b4ee4a2758 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:18.881173 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1248/2490] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:11:18 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:18.882777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1c7af-e1dd-41d7-bccf-1b8a1bdb0dd0 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:18.893477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1c7af-e1dd-41d7-bccf-1b8a1bdb0dd0 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:18.898359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861746d-861d-4b26-b524-16080cb28066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:18.899048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861746d-861d-4b26-b524-16080cb28066 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.899332 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861746d-861d-4b26-b524-16080cb28066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.899608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861746d-861d-4b26-b524-16080cb28066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.900334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861746d-861d-4b26-b524-16080cb28066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:18.901962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861746d-861d-4b26-b524-16080cb28066 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:18.907504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:18.919100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1243/2491] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:11:18.923103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6f1be-fd72-4b27-b44d-11b4ee4a2758 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:18.934021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6f1be-fd72-4b27-b44d-11b4ee4a2758 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:18.938807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf09dde-3bfc-461b-9803-4c8dde4815d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:18.939478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf09dde-3bfc-461b-9803-4c8dde4815d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.939766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf09dde-3bfc-461b-9803-4c8dde4815d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.940045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf09dde-3bfc-461b-9803-4c8dde4815d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.940825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf09dde-3bfc-461b-9803-4c8dde4815d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:18.942459 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf09dde-3bfc-461b-9803-4c8dde4815d8 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:18.956353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1249/2492] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:18.959532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861746d-861d-4b26-b524-16080cb28066 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:18.971066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861746d-861d-4b26-b524-16080cb28066 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:18.978571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71ec49-d29e-4bcd-89a2-191a82209d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:18.979880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71ec49-d29e-4bcd-89a2-191a82209d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:18.980326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71ec49-d29e-4bcd-89a2-191a82209d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:18.981102 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71ec49-d29e-4bcd-89a2-191a82209d49 admin admin] PATH_INFO: `/v3/projects/e7b0d088c6ff428099d0e771057d7465/users/834f407b5432492cb4c0b388dff0a3d0/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:18.999374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71ec49-d29e-4bcd-89a2-191a82209d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b0d088c6ff428099d0e771057d7465, user_id=834f407b5432492cb4c0b388dff0a3d0, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:19.002194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71ec49-d29e-4bcd-89a2-191a82209d49 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:19.015451 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1244/2493] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:18 2022] PUT /identity/v3/projects/e7b0d088c6ff428099d0e771057d7465/users/834f407b5432492cb4c0b388dff0a3d0/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:19.025649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3eeb9-5165-4464-a784-97bbb6f78333 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:19.025951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3eeb9-5165-4464-a784-97bbb6f78333 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:19.026219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3eeb9-5165-4464-a784-97bbb6f78333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:19.054417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-30b3eeb9-5165-4464-a784-97bbb6f78333 None None] MFA Rules not processed for user `834f407b5432492cb4c0b388dff0a3d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:19.055409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-30b3eeb9-5165-4464-a784-97bbb6f78333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:19.125872 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1250/2494] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:19.137071 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71ec49-d29e-4bcd-89a2-191a82209d49 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:19.138848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9c71ec49-d29e-4bcd-89a2-191a82209d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:19.205279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71ec49-d29e-4bcd-89a2-191a82209d49 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:19.209253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62867219-71fb-4af9-8aa8-7cc6d77b3ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:19.210457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867219-71fb-4af9-8aa8-7cc6d77b3ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:19.210938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867219-71fb-4af9-8aa8-7cc6d77b3ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:19.211225 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867219-71fb-4af9-8aa8-7cc6d77b3ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:19.219929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62867219-71fb-4af9-8aa8-7cc6d77b3ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:19.221852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62867219-71fb-4af9-8aa8-7cc6d77b3ba2 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:19.232151 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1245/2495] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:19.345360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3eeb9-5165-4464-a784-97bbb6f78333 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:19.346901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-30b3eeb9-5165-4464-a784-97bbb6f78333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:19.443321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3eeb9-5165-4464-a784-97bbb6f78333 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:19.449137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb06106-f9e6-4416-8dcc-f9b6374528e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:19.449783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06106-f9e6-4416-8dcc-f9b6374528e1 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:19.450088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06106-f9e6-4416-8dcc-f9b6374528e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:19.450362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06106-f9e6-4416-8dcc-f9b6374528e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:19.452245 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-cfb06106-f9e6-4416-8dcc-f9b6374528e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:19.465724 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb06106-f9e6-4416-8dcc-f9b6374528e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:19.467426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb06106-f9e6-4416-8dcc-f9b6374528e1 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:19.472631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:19.480464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1251/2496] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:29.227770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:29.482812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:35.107752 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce2335-2d96-407f-8193-857580c4beeb None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:35.108231 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce2335-2d96-407f-8193-857580c4beeb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:35.108343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce2335-2d96-407f-8193-857580c4beeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:35.125930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3cce2335-2d96-407f-8193-857580c4beeb None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:35.127240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3cce2335-2d96-407f-8193-857580c4beeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:35.181609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1246/2497] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:11:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:35.227251 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb06106-f9e6-4416-8dcc-f9b6374528e1 service nova] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:35.228692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-cfb06106-f9e6-4416-8dcc-f9b6374528e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:35.302469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb06106-f9e6-4416-8dcc-f9b6374528e1 service nova] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:35.307488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d4751-54c6-4281-8a79-59c494c868fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:35.308142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d4751-54c6-4281-8a79-59c494c868fb service swift] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:35.308460 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d4751-54c6-4281-8a79-59c494c868fb service swift] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:35.308744 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d4751-54c6-4281-8a79-59c494c868fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:35.317064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d4751-54c6-4281-8a79-59c494c868fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:35.318662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d4751-54c6-4281-8a79-59c494c868fb service swift] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:35.329126 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1252/2498] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:11:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 08 02:11:37.556778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce2335-2d96-407f-8193-857580c4beeb None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:37.641181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce2335-2d96-407f-8193-857580c4beeb None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:37.645858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:37.646833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:37.647877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:37.648454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] PATH_INFO: `/v3/users/834f407b5432492cb4c0b388dff0a3d0` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:37.650344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=834f407b5432492cb4c0b388dff0a3d0)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:37.652386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:37.707853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834f407b5432492cb4c0b388dff0a3d0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:37.708536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834f407b5432492cb4c0b388dff0a3d0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:37.718396 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834f407b5432492cb4c0b388dff0a3d0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:37.719193 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834f407b5432492cb4c0b388dff0a3d0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:37.729853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834f407b5432492cb4c0b388dff0a3d0'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:37.751026 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1247/2499] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:11:37 2022] DELETE /identity/v3/users/834f407b5432492cb4c0b388dff0a3d0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:37.755361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e14aa-56da-4213-a70f-259f4b3b391c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:37.755870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e14aa-56da-4213-a70f-259f4b3b391c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:37.755975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e14aa-56da-4213-a70f-259f4b3b391c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:37.777885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-fc9e14aa-56da-4213-a70f-259f4b3b391c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:37.779105 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-fc9e14aa-56da-4213-a70f-259f4b3b391c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:37.856664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1253/2500] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:37.867410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:37.869632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:37.908262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:37.972131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c6938-619d-43ea-aa50-5c6a89a9abc8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:37.979459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32292827-630a-4c7e-aa68-731c4cd26dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:37.980150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292827-630a-4c7e-aa68-731c4cd26dfe service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:37.980478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292827-630a-4c7e-aa68-731c4cd26dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:37.980790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292827-630a-4c7e-aa68-731c4cd26dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:37.993005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292827-630a-4c7e-aa68-731c4cd26dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:37.994037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292827-630a-4c7e-aa68-731c4cd26dfe service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:38.007299 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1248/2501] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:38.384859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e14aa-56da-4213-a70f-259f4b3b391c None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:38.386441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-fc9e14aa-56da-4213-a70f-259f4b3b391c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:38.401706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e14aa-56da-4213-a70f-259f4b3b391c None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:38.407881 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:38.409076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:38.409467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:38.409847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] PATH_INFO: `/v3/projects/e7b0d088c6ff428099d0e771057d7465` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:38.411997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b0d088c6ff428099d0e771057d7465)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:38.414777 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:38.486140 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b0d088c6ff428099d0e771057d7465', 'request_id': 'req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:38.495367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b0d088c6ff428099d0e771057d7465', 'request_id': 'req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:38.502384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b0d088c6ff428099d0e771057d7465', 'request_id': 'req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:38.517769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] The token cache is being invalidate because project e7b0d088c6ff428099d0e771057d7465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:11:38.518205 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:38.518934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-c02a672c-7737-4af5-ac0c-d50d9da9bc3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:11:38.523826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1254/2502] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:11:38 2022] DELETE /identity/v3/projects/e7b0d088c6ff428099d0e771057d7465 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:38.529132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75418b1-1c08-48f6-badf-7e744d5e2b45 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:38.529425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75418b1-1c08-48f6-badf-7e744d5e2b45 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:38.529702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75418b1-1c08-48f6-badf-7e744d5e2b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:38.553647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b75418b1-1c08-48f6-badf-7e744d5e2b45 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:38.555332 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b75418b1-1c08-48f6-badf-7e744d5e2b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:38.642278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1249/2503] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:38.643840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f838fba-1425-40a6-a796-e15f10763dd3 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:38.644129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f838fba-1425-40a6-a796-e15f10763dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:38.644420 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f838fba-1425-40a6-a796-e15f10763dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:38.671856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-6f838fba-1425-40a6-a796-e15f10763dd3 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:38.673101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6f838fba-1425-40a6-a796-e15f10763dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:38.684973 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1255/2504] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:38.685949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75418b1-1c08-48f6-badf-7e744d5e2b45 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:38.696373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75418b1-1c08-48f6-badf-7e744d5e2b45 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:38.704250 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1f89d-259b-41a9-a06f-a0a1a82e7442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:38.704978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1f89d-259b-41a9-a06f-a0a1a82e7442 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:38.705265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1f89d-259b-41a9-a06f-a0a1a82e7442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:38.705544 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1f89d-259b-41a9-a06f-a0a1a82e7442 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:38.706305 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1f89d-259b-41a9-a06f-a0a1a82e7442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:11:38.706619 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1f89d-259b-41a9-a06f-a0a1a82e7442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:38.708237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1f89d-259b-41a9-a06f-a0a1a82e7442 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:38.724930 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1250/2505] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 02:11:38.726465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f838fba-1425-40a6-a796-e15f10763dd3 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:38.736753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f838fba-1425-40a6-a796-e15f10763dd3 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:38.743191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48953e2-e5b7-41be-bbc7-991259b107a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:38.743861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48953e2-e5b7-41be-bbc7-991259b107a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:38.744141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48953e2-e5b7-41be-bbc7-991259b107a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:38.744439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48953e2-e5b7-41be-bbc7-991259b107a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:38.745130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48953e2-e5b7-41be-bbc7-991259b107a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:38.747034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48953e2-e5b7-41be-bbc7-991259b107a9 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:38.750673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:38.780249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1256/2506] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:11:38 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:11:38.784975 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1f89d-259b-41a9-a06f-a0a1a82e7442 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:38.798071 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1f89d-259b-41a9-a06f-a0a1a82e7442 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:38.872371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0c30d-4871-4227-bee7-6637396cfca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:38.873826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0c30d-4871-4227-bee7-6637396cfca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:38.874145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0c30d-4871-4227-bee7-6637396cfca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:38.874502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0c30d-4871-4227-bee7-6637396cfca1 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:38.875372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0c30d-4871-4227-bee7-6637396cfca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:38.877842 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0c30d-4871-4227-bee7-6637396cfca1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:38.903723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1251/2507] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:11:38 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:38.904931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48953e2-e5b7-41be-bbc7-991259b107a9 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:38.918311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48953e2-e5b7-41be-bbc7-991259b107a9 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:38.924823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df82f0-aab0-4ae9-89e7-624bab20b52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:38.925511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df82f0-aab0-4ae9-89e7-624bab20b52c admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:38.925962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df82f0-aab0-4ae9-89e7-624bab20b52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:38.926250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df82f0-aab0-4ae9-89e7-624bab20b52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:38.926997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df82f0-aab0-4ae9-89e7-624bab20b52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:38.929247 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df82f0-aab0-4ae9-89e7-624bab20b52c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:38.940175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1257/2508] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:11:38.944656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0c30d-4871-4227-bee7-6637396cfca1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:38.955475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0c30d-4871-4227-bee7-6637396cfca1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:38.962876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7d11b-a98b-4e71-bdd7-c5161510fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:38.962876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7d11b-a98b-4e71-bdd7-c5161510fca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:38.962876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7d11b-a98b-4e71-bdd7-c5161510fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:38.962876 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7d11b-a98b-4e71-bdd7-c5161510fca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:38.964048 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7d11b-a98b-4e71-bdd7-c5161510fca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:38.964287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7d11b-a98b-4e71-bdd7-c5161510fca9 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:38.982272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1252/2509] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:38.986454 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df82f0-aab0-4ae9-89e7-624bab20b52c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:38.998665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df82f0-aab0-4ae9-89e7-624bab20b52c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:39.003544 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e8021-7925-4c69-b0f8-e954ca29f699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:39.004891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e8021-7925-4c69-b0f8-e954ca29f699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:39.005168 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e8021-7925-4c69-b0f8-e954ca29f699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:39.005987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e8021-7925-4c69-b0f8-e954ca29f699 admin admin] PATH_INFO: `/v3/projects/3b0724451ec94fdbb4f2422c44245b53/users/2f0ae440fa644f1296d2b474c5335ffc/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:39.026828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e8021-7925-4c69-b0f8-e954ca29f699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0724451ec94fdbb4f2422c44245b53, user_id=2f0ae440fa644f1296d2b474c5335ffc, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:39.030302 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e8021-7925-4c69-b0f8-e954ca29f699 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:39.049224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1258/2510] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:38 2022] PUT /identity/v3/projects/3b0724451ec94fdbb4f2422c44245b53/users/2f0ae440fa644f1296d2b474c5335ffc/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:39.055060 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350356ca-90f3-4eb3-b6d5-6dd3efc7efba None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:39.055362 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350356ca-90f3-4eb3-b6d5-6dd3efc7efba None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:39.055671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350356ca-90f3-4eb3-b6d5-6dd3efc7efba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:39.076003 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-350356ca-90f3-4eb3-b6d5-6dd3efc7efba None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:39.078872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-350356ca-90f3-4eb3-b6d5-6dd3efc7efba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:39.126086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:39.176729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1253/2511] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:39.187660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e8021-7925-4c69-b0f8-e954ca29f699 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:39.189626 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a80e8021-7925-4c69-b0f8-e954ca29f699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:39.269202 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e8021-7925-4c69-b0f8-e954ca29f699 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:39.273966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b854d-ac7b-4588-a0a3-a21137826232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:39.274630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b854d-ac7b-4588-a0a3-a21137826232 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:39.274911 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b854d-ac7b-4588-a0a3-a21137826232 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:39.275184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b854d-ac7b-4588-a0a3-a21137826232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:39.286337 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b854d-ac7b-4588-a0a3-a21137826232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:39.288103 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b854d-ac7b-4588-a0a3-a21137826232 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:39.300056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1259/2512] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:45.338419 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885998b-d504-4d5b-ab2d-2a0a7031a270 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:45.338957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885998b-d504-4d5b-ab2d-2a0a7031a270 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:45.338957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885998b-d504-4d5b-ab2d-2a0a7031a270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:45.403012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8885998b-d504-4d5b-ab2d-2a0a7031a270 None None] MFA Rules not processed for user `2f0ae440fa644f1296d2b474c5335ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:45.404427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8885998b-d504-4d5b-ab2d-2a0a7031a270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:45.490586 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1254/2513] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:45.495854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b854d-ac7b-4588-a0a3-a21137826232 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:45.514550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b854d-ac7b-4588-a0a3-a21137826232 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:45.520371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352c2aa-740d-4212-ba23-8bbbabf07679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:45.521107 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352c2aa-740d-4212-ba23-8bbbabf07679 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:45.521397 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352c2aa-740d-4212-ba23-8bbbabf07679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:45.521741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352c2aa-740d-4212-ba23-8bbbabf07679 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:45.523354 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352c2aa-740d-4212-ba23-8bbbabf07679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:45.525479 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352c2aa-740d-4212-ba23-8bbbabf07679 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:45.576852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1260/2514] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:11:45 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:45.580685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885998b-d504-4d5b-ab2d-2a0a7031a270 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:45.599872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885998b-d504-4d5b-ab2d-2a0a7031a270 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:45.718460 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925803a4-ee15-49f6-8c09-f39c0b3c02d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:45.719613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925803a4-ee15-49f6-8c09-f39c0b3c02d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:45.719613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925803a4-ee15-49f6-8c09-f39c0b3c02d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:45.719816 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925803a4-ee15-49f6-8c09-f39c0b3c02d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:45.720519 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925803a4-ee15-49f6-8c09-f39c0b3c02d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:45.722626 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925803a4-ee15-49f6-8c09-f39c0b3c02d5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:45.752825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1255/2515] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:11:45 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:11:45.757014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352c2aa-740d-4212-ba23-8bbbabf07679 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:45.771956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352c2aa-740d-4212-ba23-8bbbabf07679 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:45.777520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0061f99-3e28-48bc-976b-31b95f7b6ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:45.778198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0061f99-3e28-48bc-976b-31b95f7b6ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:45.780349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0061f99-3e28-48bc-976b-31b95f7b6ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:45.780349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0061f99-3e28-48bc-976b-31b95f7b6ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:45.780349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0061f99-3e28-48bc-976b-31b95f7b6ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:45.782889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0061f99-3e28-48bc-976b-31b95f7b6ba2 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:45.796870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1261/2516] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:45.799783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925803a4-ee15-49f6-8c09-f39c0b3c02d5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:45.811283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925803a4-ee15-49f6-8c09-f39c0b3c02d5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:45.816169 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51fcaa-1013-4b3b-ab73-5136a74704a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:45.817215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51fcaa-1013-4b3b-ab73-5136a74704a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:45.817593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51fcaa-1013-4b3b-ab73-5136a74704a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:45.817968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51fcaa-1013-4b3b-ab73-5136a74704a6 admin admin] PATH_INFO: `/v3/projects/3424f51f284c42338dc4295f2860a4a4/users/960a45d3bb3c4c2cbec522cd11d70d90/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:45.838084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51fcaa-1013-4b3b-ab73-5136a74704a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3424f51f284c42338dc4295f2860a4a4, user_id=960a45d3bb3c4c2cbec522cd11d70d90, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:45.840820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51fcaa-1013-4b3b-ab73-5136a74704a6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:45.859426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1256/2517] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:45 2022] PUT /identity/v3/projects/3424f51f284c42338dc4295f2860a4a4/users/960a45d3bb3c4c2cbec522cd11d70d90/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:45.863226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0061f99-3e28-48bc-976b-31b95f7b6ba2 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:45.876944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0061f99-3e28-48bc-976b-31b95f7b6ba2 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:45.968194 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87317627-550f-4bda-80e4-a79b73137a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:45.969115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87317627-550f-4bda-80e4-a79b73137a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:45.969464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87317627-550f-4bda-80e4-a79b73137a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:45.969807 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87317627-550f-4bda-80e4-a79b73137a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:45.970608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87317627-550f-4bda-80e4-a79b73137a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:45.972414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87317627-550f-4bda-80e4-a79b73137a45 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:45.988387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1262/2518] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:11:45.989193 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51fcaa-1013-4b3b-ab73-5136a74704a6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:46.024248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51fcaa-1013-4b3b-ab73-5136a74704a6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:46.029137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6d0b3-3f05-4b4e-a509-153cc89f34d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:46.030049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6d0b3-3f05-4b4e-a509-153cc89f34d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:46.030338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6d0b3-3f05-4b4e-a509-153cc89f34d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:46.030617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6d0b3-3f05-4b4e-a509-153cc89f34d3 admin admin] PATH_INFO: `/v3/projects/3424f51f284c42338dc4295f2860a4a4/users/960a45d3bb3c4c2cbec522cd11d70d90/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:46.032893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6d0b3-3f05-4b4e-a509-153cc89f34d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3424f51f284c42338dc4295f2860a4a4, user_id=960a45d3bb3c4c2cbec522cd11d70d90, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:46.035571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6d0b3-3f05-4b4e-a509-153cc89f34d3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:46.050697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1257/2519] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:11:45 2022] PUT /identity/v3/projects/3424f51f284c42338dc4295f2860a4a4/users/960a45d3bb3c4c2cbec522cd11d70d90/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:11:49.882897 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87317627-550f-4bda-80e4-a79b73137a45 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:49.884952 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-87317627-550f-4bda-80e4-a79b73137a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:49.900254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:50.040963 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87317627-550f-4bda-80e4-a79b73137a45 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:50.200499 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a06e-fa8e-43f9-9ce8-0f2701a2f1fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:50.201387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a06e-fa8e-43f9-9ce8-0f2701a2f1fb service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:50.201743 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a06e-fa8e-43f9-9ce8-0f2701a2f1fb service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:50.202088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a06e-fa8e-43f9-9ce8-0f2701a2f1fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:50.204218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7391a06e-fa8e-43f9-9ce8-0f2701a2f1fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:50.333271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391a06e-fa8e-43f9-9ce8-0f2701a2f1fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:50.335887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391a06e-fa8e-43f9-9ce8-0f2701a2f1fb service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:50.348728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1263/2520] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:11:51.236773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1483-a65a-4ecc-a895-ed83a4f885ed None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:51.237744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1483-a65a-4ecc-a895-ed83a4f885ed None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:51.238238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1483-a65a-4ecc-a895-ed83a4f885ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:51.270540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6dbb1483-a65a-4ecc-a895-ed83a4f885ed None None] MFA Rules not processed for user `960a45d3bb3c4c2cbec522cd11d70d90`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:51.272713 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6dbb1483-a65a-4ecc-a895-ed83a4f885ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:51.277669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:51.360699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1258/2521] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:11:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:51.377019 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a06e-fa8e-43f9-9ce8-0f2701a2f1fb service nova] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:51.378844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7391a06e-fa8e-43f9-9ce8-0f2701a2f1fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:51.452983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:11:51.467180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a06e-fa8e-43f9-9ce8-0f2701a2f1fb service nova] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:51.471603 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029958e9-3a6c-4fdb-9515-caedcb0e73a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d1e01f6f82a4ebbb15d0cb55b963af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:51.472283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029958e9-3a6c-4fdb-9515-caedcb0e73a3 service glance] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:51.473208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029958e9-3a6c-4fdb-9515-caedcb0e73a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:51.473506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029958e9-3a6c-4fdb-9515-caedcb0e73a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:51.556685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029958e9-3a6c-4fdb-9515-caedcb0e73a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:51.558318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029958e9-3a6c-4fdb-9515-caedcb0e73a3 service glance] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:51.572958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1264/2522] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:51.709442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec2a12-1f12-419c-8712-88f0048901b9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:51.710112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec2a12-1f12-419c-8712-88f0048901b9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:51.710719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec2a12-1f12-419c-8712-88f0048901b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:51.734624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e7ec2a12-1f12-419c-8712-88f0048901b9 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:51.749191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e7ec2a12-1f12-419c-8712-88f0048901b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:51.833966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1259/2523] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:11:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:51.855979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029958e9-3a6c-4fdb-9515-caedcb0e73a3 service glance] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:51.857631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-029958e9-3a6c-4fdb-9515-caedcb0e73a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:51.963948 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029958e9-3a6c-4fdb-9515-caedcb0e73a3 service glance] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:51.969605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945b123-a991-40dc-bd7b-d96a9775eb78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:51.970289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945b123-a991-40dc-bd7b-d96a9775eb78 service swift] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:51.970709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945b123-a991-40dc-bd7b-d96a9775eb78 service swift] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:51.971023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945b123-a991-40dc-bd7b-d96a9775eb78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:51.981569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945b123-a991-40dc-bd7b-d96a9775eb78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:51.983236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945b123-a991-40dc-bd7b-d96a9775eb78 service swift] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:51.993396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1265/2524] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:11:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:11:52.340609 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389649a5-ea18-42d7-88f7-b9cfff1a465f None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:52.341168 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389649a5-ea18-42d7-88f7-b9cfff1a465f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:52.341168 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389649a5-ea18-42d7-88f7-b9cfff1a465f None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:52.343769 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1260/2525] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:11:52.350973 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b2471-5078-401b-9302-691e57b64bae None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:52.351406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b2471-5078-401b-9302-691e57b64bae None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:52.351727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b2471-5078-401b-9302-691e57b64bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:52.392699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7d8b2471-5078-401b-9302-691e57b64bae None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:52.393939 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7d8b2471-5078-401b-9302-691e57b64bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:52.434071 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1266/2526] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:11:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:52.448357 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389649a5-ea18-42d7-88f7-b9cfff1a465f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:52.465976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389649a5-ea18-42d7-88f7-b9cfff1a465f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:52.471995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c6c26-bd4f-4ce8-958e-5afe75035d28 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d1e01f6f82a4ebbb15d0cb55b963af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:52.472673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c6c26-bd4f-4ce8-958e-5afe75035d28 service glance] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:52.472954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c6c26-bd4f-4ce8-958e-5afe75035d28 service glance] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:52.473228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c6c26-bd4f-4ce8-958e-5afe75035d28 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:52.504076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c6c26-bd4f-4ce8-958e-5afe75035d28 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:52.507025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c6c26-bd4f-4ce8-958e-5afe75035d28 service glance] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:52.521536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1261/2527] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:52.606051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d888b-7977-463e-b384-15a00c5b5d33 None None] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:52.606665 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d888b-7977-463e-b384-15a00c5b5d33 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:52.607034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d888b-7977-463e-b384-15a00c5b5d33 None None] PATH_INFO: `/` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:52.608491 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1267/2528] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:11:52.614015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5f4b-198e-4302-9097-154eebc4e919 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:52.614310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5f4b-198e-4302-9097-154eebc4e919 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:52.614682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5f4b-198e-4302-9097-154eebc4e919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:52.636608 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-302f5f4b-198e-4302-9097-154eebc4e919 None None] MFA Rules not processed for user `d26f4426ba8c43399c4a7ef2f1701f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:52.638264 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-302f5f4b-198e-4302-9097-154eebc4e919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:52.701410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1262/2529] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:11:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:11:52.714838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d888b-7977-463e-b384-15a00c5b5d33 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:52.728033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d888b-7977-463e-b384-15a00c5b5d33 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:52.731667 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b7295-e331-4681-9c78-0965b02a4dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:52.732588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7295-e331-4681-9c78-0965b02a4dda service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:52.732884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7295-e331-4681-9c78-0965b02a4dda service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:52.733241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7295-e331-4681-9c78-0965b02a4dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:52.743510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b7295-e331-4681-9c78-0965b02a4dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:52.745079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b7295-e331-4681-9c78-0965b02a4dda service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:52.765786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1268/2530] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:52.856737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfdd5f-9309-4bb1-b655-bb6fa5d3d139 None None] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:52.857032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfdd5f-9309-4bb1-b655-bb6fa5d3d139 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:52.857308 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfdd5f-9309-4bb1-b655-bb6fa5d3d139 None None] PATH_INFO: `/` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:52.859351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1263/2531] 10.222.0.20 () {54 vars in 921 bytes} [Wed Jun 8 02:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 08 02:11:52.870606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989964-3281-46ba-bf42-9a1ca8593017 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:52.871308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989964-3281-46ba-bf42-9a1ca8593017 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:52.871600 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989964-3281-46ba-bf42-9a1ca8593017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:52.914895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7b989964-3281-46ba-bf42-9a1ca8593017 None None] MFA Rules not processed for user `0ef930dc7b7a4fb29f7b32e464e9eb3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:52.917117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7b989964-3281-46ba-bf42-9a1ca8593017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:52.940365 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1269/2532] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Jun 8 02:11:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:52.954106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfdd5f-9309-4bb1-b655-bb6fa5d3d139 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:52.968549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfdd5f-9309-4bb1-b655-bb6fa5d3d139 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:52.974739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1483097-f732-468d-82dc-130fba306fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:52.976224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1483097-f732-468d-82dc-130fba306fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:52.976818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1483097-f732-468d-82dc-130fba306fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:52.977331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1483097-f732-468d-82dc-130fba306fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:52.994064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1483097-f732-468d-82dc-130fba306fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:52.996575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1483097-f732-468d-82dc-130fba306fa2 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:53.025462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1264/2533] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:11:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:11:55.310690 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a73c-292c-457e-9ca8-68245671f503 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:55.311571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a73c-292c-457e-9ca8-68245671f503 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:55.311855 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a73c-292c-457e-9ca8-68245671f503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:55.346486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-2e70a73c-292c-457e-9ca8-68245671f503 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:11:55.348084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2e70a73c-292c-457e-9ca8-68245671f503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:11:55.358951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1270/2534] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:11:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:11:55.379398 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1483097-f732-468d-82dc-130fba306fa2 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:11:55.403853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1483097-f732-468d-82dc-130fba306fa2 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:11:55.410526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d3313-7725-4aa3-a9a9-3f5120b78339 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:11:55.411371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d3313-7725-4aa3-a9a9-3f5120b78339 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:11:55.411461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d3313-7725-4aa3-a9a9-3f5120b78339 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:11:55.411730 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d3313-7725-4aa3-a9a9-3f5120b78339 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:11:55.421297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d3313-7725-4aa3-a9a9-3f5120b78339 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:11:55.422914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d3313-7725-4aa3-a9a9-3f5120b78339 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:11:55.439078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1265/2535] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:11:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 08 02:12:03.473414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99770ab-10c1-485b-94e9-aab950aee932 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:03.473972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99770ab-10c1-485b-94e9-aab950aee932 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:03.474079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99770ab-10c1-485b-94e9-aab950aee932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:03.492853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e99770ab-10c1-485b-94e9-aab950aee932 None None] MFA Rules not processed for user `2f0ae440fa644f1296d2b474c5335ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:03.494055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e99770ab-10c1-485b-94e9-aab950aee932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:03.500884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1271/2536] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:03.777374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d3313-7725-4aa3-a9a9-3f5120b78339 service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:03.778829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1b6d3313-7725-4aa3-a9a9-3f5120b78339 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:03.792238 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d3313-7725-4aa3-a9a9-3f5120b78339 service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:03.795887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4013f-b27d-40b1-baf4-a49fe39d17ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:03.796572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4013f-b27d-40b1-baf4-a49fe39d17ea service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:03.796860 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4013f-b27d-40b1-baf4-a49fe39d17ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:03.797137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4013f-b27d-40b1-baf4-a49fe39d17ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:03.799018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4ba4013f-b27d-40b1-baf4-a49fe39d17ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:03.809976 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4013f-b27d-40b1-baf4-a49fe39d17ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:03.811435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4013f-b27d-40b1-baf4-a49fe39d17ea service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:03.820062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1266/2537] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:06.929924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e2674-7360-4ce0-9ad7-8530e27fa619 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:06.930411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e2674-7360-4ce0-9ad7-8530e27fa619 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:06.930787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e2674-7360-4ce0-9ad7-8530e27fa619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:06.949258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-cb4e2674-7360-4ce0-9ad7-8530e27fa619 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:06.950568 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-cb4e2674-7360-4ce0-9ad7-8530e27fa619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:06.959166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1272/2538] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:12:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:06.971566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4013f-b27d-40b1-baf4-a49fe39d17ea service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:06.978077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:06.984965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4013f-b27d-40b1-baf4-a49fe39d17ea service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:06.990201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eeff6a-2c7d-41e3-93bb-46f0723c1b6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:06.990856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eeff6a-2c7d-41e3-93bb-46f0723c1b6c service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:06.991147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eeff6a-2c7d-41e3-93bb-46f0723c1b6c service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:06.992063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eeff6a-2c7d-41e3-93bb-46f0723c1b6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:07.000778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eeff6a-2c7d-41e3-93bb-46f0723c1b6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:07.002515 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eeff6a-2c7d-41e3-93bb-46f0723c1b6c service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:07.011947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1267/2539] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:12:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:12:16.943206 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=82 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:17.008378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:17.448526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6ca9b-d6a9-438e-a382-5c1f31dfbd6e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:17.448830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6ca9b-d6a9-438e-a382-5c1f31dfbd6e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:17.449116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6ca9b-d6a9-438e-a382-5c1f31dfbd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:17.466987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-05c6ca9b-d6a9-438e-a382-5c1f31dfbd6e None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:17.468321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-05c6ca9b-d6a9-438e-a382-5c1f31dfbd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:17.475790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1273/2540] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jun 8 02:12:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:17.489471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eeff6a-2c7d-41e3-93bb-46f0723c1b6c service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:17.501857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eeff6a-2c7d-41e3-93bb-46f0723c1b6c service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:17.507187 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c808da-5e71-4abd-9b68-6e3f0b7080fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:17.508847 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c808da-5e71-4abd-9b68-6e3f0b7080fc service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:17.509139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c808da-5e71-4abd-9b68-6e3f0b7080fc service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:17.509421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c808da-5e71-4abd-9b68-6e3f0b7080fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:17.518301 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c808da-5e71-4abd-9b68-6e3f0b7080fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:17.519887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c808da-5e71-4abd-9b68-6e3f0b7080fc service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:17.529201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1268/2541] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:12:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:12:27.461618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:27.526217 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:29.245089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6ca9b-d6a9-438e-a382-5c1f31dfbd6e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:29.254510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6ca9b-d6a9-438e-a382-5c1f31dfbd6e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:29.258150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c12d0e-bd3b-449e-9efb-aeb709520cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:29.258793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c12d0e-bd3b-449e-9efb-aeb709520cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:29.259080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c12d0e-bd3b-449e-9efb-aeb709520cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:29.259356 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c12d0e-bd3b-449e-9efb-aeb709520cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:29.261148 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b3c12d0e-bd3b-449e-9efb-aeb709520cf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:29.324528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c12d0e-bd3b-449e-9efb-aeb709520cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:29.325979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c12d0e-bd3b-449e-9efb-aeb709520cf4 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:29.336282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1274/2542] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:32.149311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c808da-5e71-4abd-9b68-6e3f0b7080fc service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:32.150756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-22c808da-5e71-4abd-9b68-6e3f0b7080fc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:32.166116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c808da-5e71-4abd-9b68-6e3f0b7080fc service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:32.170952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:32.171826 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:32.172108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:32.172381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] PATH_INFO: `/v3/users/1b43d2b8b9084099b832332a565e741c` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:32.173599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b43d2b8b9084099b832332a565e741c)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:32.175681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:32.221719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b43d2b8b9084099b832332a565e741c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:32.222444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b43d2b8b9084099b832332a565e741c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:32.230210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b43d2b8b9084099b832332a565e741c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:32.230838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b43d2b8b9084099b832332a565e741c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:32.238810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-0b4c7e04-1de3-4e1c-95d0-b27c71e0916e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b43d2b8b9084099b832332a565e741c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:32.261070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1269/2543] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:12:32 2022] DELETE /identity/v3/users/1b43d2b8b9084099b832332a565e741c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:32.433494 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c12d0e-bd3b-449e-9efb-aeb709520cf4 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:32.443588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c12d0e-bd3b-449e-9efb-aeb709520cf4 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:32.502226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:32.503129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:32.503449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:32.503801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] PATH_INFO: `/v3/projects/27f98152ab3440cf824c9ceb1c790bcf` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:32.505083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f98152ab3440cf824c9ceb1c790bcf)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:32.507150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:32.552694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f98152ab3440cf824c9ceb1c790bcf', 'request_id': 'req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:32.559340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f98152ab3440cf824c9ceb1c790bcf', 'request_id': 'req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:32.565497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f98152ab3440cf824c9ceb1c790bcf', 'request_id': 'req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:32.579892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] The token cache is being invalidate because project 27f98152ab3440cf824c9ceb1c790bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:12:32.580179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:32.581386 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-93ea1dd3-5600-45ed-b379-aba7c7c4c0dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:32.582924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1275/2544] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:12:32 2022] DELETE /identity/v3/projects/27f98152ab3440cf824c9ceb1c790bcf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:32.588605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7f8ba-59b2-4da2-8d9d-640ad5acbbf9 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:32.588921 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7f8ba-59b2-4da2-8d9d-640ad5acbbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:32.589202 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7f8ba-59b2-4da2-8d9d-640ad5acbbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:32.609436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-78b7f8ba-59b2-4da2-8d9d-640ad5acbbf9 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:32.610575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-78b7f8ba-59b2-4da2-8d9d-640ad5acbbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:32.668725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1270/2545] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:32.677275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c822d1-ddb4-43b8-865b-58787691d97e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:32.677571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c822d1-ddb4-43b8-865b-58787691d97e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:32.677854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c822d1-ddb4-43b8-865b-58787691d97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:32.697559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-83c822d1-ddb4-43b8-865b-58787691d97e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:32.698753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-83c822d1-ddb4-43b8-865b-58787691d97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:32.706551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1276/2546] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:32.711209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7f8ba-59b2-4da2-8d9d-640ad5acbbf9 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:32.720729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7f8ba-59b2-4da2-8d9d-640ad5acbbf9 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:32.725430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b5d21-d5e8-431e-b18d-7758d93fe59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:32.726089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b5d21-d5e8-431e-b18d-7758d93fe59f admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:32.726376 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b5d21-d5e8-431e-b18d-7758d93fe59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:32.726656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b5d21-d5e8-431e-b18d-7758d93fe59f admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:32.727375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b5d21-d5e8-431e-b18d-7758d93fe59f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:12:32.727679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b5d21-d5e8-431e-b18d-7758d93fe59f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:32.729353 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b5d21-d5e8-431e-b18d-7758d93fe59f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:32.740959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1271/2547] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 02:12:32.745269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c822d1-ddb4-43b8-865b-58787691d97e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:32.754429 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c822d1-ddb4-43b8-865b-58787691d97e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:32.759741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00363b16-bb38-4c05-b448-3f21eb7ad482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:32.760415 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00363b16-bb38-4c05-b448-3f21eb7ad482 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:32.760703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00363b16-bb38-4c05-b448-3f21eb7ad482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:32.761014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00363b16-bb38-4c05-b448-3f21eb7ad482 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:32.761715 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00363b16-bb38-4c05-b448-3f21eb7ad482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:32.763742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00363b16-bb38-4c05-b448-3f21eb7ad482 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:32.779749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:32.796469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1277/2548] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:12:32 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:32.797901 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b5d21-d5e8-431e-b18d-7758d93fe59f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:32.808260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b5d21-d5e8-431e-b18d-7758d93fe59f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:32.866490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a715517-4cf0-4469-8784-3cd95e7ff3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:32.867178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a715517-4cf0-4469-8784-3cd95e7ff3bf admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:32.867457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a715517-4cf0-4469-8784-3cd95e7ff3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:32.867729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a715517-4cf0-4469-8784-3cd95e7ff3bf admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:32.868606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a715517-4cf0-4469-8784-3cd95e7ff3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:32.870624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a715517-4cf0-4469-8784-3cd95e7ff3bf admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:32.878123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:32.893008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1272/2549] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:12:32 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:32.897257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00363b16-bb38-4c05-b448-3f21eb7ad482 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:32.907595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00363b16-bb38-4c05-b448-3f21eb7ad482 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:32.912599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadae5ea-4a14-42bf-80b4-eff90a3d49d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:32.913264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadae5ea-4a14-42bf-80b4-eff90a3d49d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:32.913552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadae5ea-4a14-42bf-80b4-eff90a3d49d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:32.914498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadae5ea-4a14-42bf-80b4-eff90a3d49d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:32.915232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadae5ea-4a14-42bf-80b4-eff90a3d49d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:32.916888 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadae5ea-4a14-42bf-80b4-eff90a3d49d4 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:32.927680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1278/2550] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:32.932464 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a715517-4cf0-4469-8784-3cd95e7ff3bf admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:32.942399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a715517-4cf0-4469-8784-3cd95e7ff3bf admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:32.947399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f163cef-9ad4-4a5b-a1a3-0f3a4701b140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:32.948052 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f163cef-9ad4-4a5b-a1a3-0f3a4701b140 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:32.948337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f163cef-9ad4-4a5b-a1a3-0f3a4701b140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:32.948646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f163cef-9ad4-4a5b-a1a3-0f3a4701b140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:32.949374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f163cef-9ad4-4a5b-a1a3-0f3a4701b140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:32.950964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f163cef-9ad4-4a5b-a1a3-0f3a4701b140 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:32.960987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1273/2551] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:32.964732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadae5ea-4a14-42bf-80b4-eff90a3d49d4 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:32.974944 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadae5ea-4a14-42bf-80b4-eff90a3d49d4 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:32.980179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa6c6e-3d9a-4d15-9d91-b5afa1da3e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:32.981083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa6c6e-3d9a-4d15-9d91-b5afa1da3e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:32.981367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa6c6e-3d9a-4d15-9d91-b5afa1da3e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:32.981642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa6c6e-3d9a-4d15-9d91-b5afa1da3e68 admin admin] PATH_INFO: `/v3/projects/b23f47cea0cd48fea5346e9331c1da49/users/6930e78c8a8b44c8a6c136b68f12c2a4/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:32.996947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa6c6e-3d9a-4d15-9d91-b5afa1da3e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23f47cea0cd48fea5346e9331c1da49, user_id=6930e78c8a8b44c8a6c136b68f12c2a4, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:32.999588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa6c6e-3d9a-4d15-9d91-b5afa1da3e68 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:33.012353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1279/2552] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:12:32 2022] PUT /identity/v3/projects/b23f47cea0cd48fea5346e9331c1da49/users/6930e78c8a8b44c8a6c136b68f12c2a4/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:33.022206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd2a5c-165e-4a92-8d6b-7f70af7a373d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:33.022504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd2a5c-165e-4a92-8d6b-7f70af7a373d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:33.022787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd2a5c-165e-4a92-8d6b-7f70af7a373d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:33.050180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-7ebd2a5c-165e-4a92-8d6b-7f70af7a373d None None] MFA Rules not processed for user `6930e78c8a8b44c8a6c136b68f12c2a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:33.051422 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7ebd2a5c-165e-4a92-8d6b-7f70af7a373d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:33.114042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1274/2553] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:33.118632 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa6c6e-3d9a-4d15-9d91-b5afa1da3e68 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:33.180484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa6c6e-3d9a-4d15-9d91-b5afa1da3e68 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:33.185035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b44ce-0638-4c08-95e8-c7b3ce6eb937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:33.185719 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b44ce-0638-4c08-95e8-c7b3ce6eb937 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:33.186008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b44ce-0638-4c08-95e8-c7b3ce6eb937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:33.186285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b44ce-0638-4c08-95e8-c7b3ce6eb937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:33.186968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b44ce-0638-4c08-95e8-c7b3ce6eb937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:33.188927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b44ce-0638-4c08-95e8-c7b3ce6eb937 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:33.214185 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1280/2554] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:12:33 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:33.218138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd2a5c-165e-4a92-8d6b-7f70af7a373d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:33.229538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd2a5c-165e-4a92-8d6b-7f70af7a373d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:33.285785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a46f1-de9e-4941-96c3-7b782ee08a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:33.286477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a46f1-de9e-4941-96c3-7b782ee08a08 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:33.286760 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a46f1-de9e-4941-96c3-7b782ee08a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:33.287033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a46f1-de9e-4941-96c3-7b782ee08a08 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:33.287723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a46f1-de9e-4941-96c3-7b782ee08a08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:33.290025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a46f1-de9e-4941-96c3-7b782ee08a08 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:33.310599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1275/2555] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:12:33 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:33.314693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b44ce-0638-4c08-95e8-c7b3ce6eb937 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:33.325764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b44ce-0638-4c08-95e8-c7b3ce6eb937 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:33.330283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6529d64-17dc-4196-9ca4-46e7716c02d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:33.330932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6529d64-17dc-4196-9ca4-46e7716c02d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:33.331212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6529d64-17dc-4196-9ca4-46e7716c02d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:33.331483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6529d64-17dc-4196-9ca4-46e7716c02d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:33.332201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6529d64-17dc-4196-9ca4-46e7716c02d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:33.333833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6529d64-17dc-4196-9ca4-46e7716c02d1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:33.344717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1281/2556] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:33.348942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a46f1-de9e-4941-96c3-7b782ee08a08 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:33.359821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a46f1-de9e-4941-96c3-7b782ee08a08 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:33.364348 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd233eb-33d9-4f77-afed-a7d7e8212040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:33.365235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd233eb-33d9-4f77-afed-a7d7e8212040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:33.365513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd233eb-33d9-4f77-afed-a7d7e8212040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:33.365785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd233eb-33d9-4f77-afed-a7d7e8212040 admin admin] PATH_INFO: `/v3/projects/121ea24896c74156b61335ef15a07e45/users/804a2984241f4d428f076e41b62bca07/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:33.380910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd233eb-33d9-4f77-afed-a7d7e8212040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121ea24896c74156b61335ef15a07e45, user_id=804a2984241f4d428f076e41b62bca07, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:33.383893 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd233eb-33d9-4f77-afed-a7d7e8212040 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:33.396369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1276/2557] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:12:33 2022] PUT /identity/v3/projects/121ea24896c74156b61335ef15a07e45/users/804a2984241f4d428f076e41b62bca07/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:12:33.400673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6529d64-17dc-4196-9ca4-46e7716c02d1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:33.411018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6529d64-17dc-4196-9ca4-46e7716c02d1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:33.467294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66347d-634f-4950-90dd-f7436f07098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:33.467958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66347d-634f-4950-90dd-f7436f07098b admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:33.468214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66347d-634f-4950-90dd-f7436f07098b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:33.468522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66347d-634f-4950-90dd-f7436f07098b admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:33.469234 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66347d-634f-4950-90dd-f7436f07098b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:33.470826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66347d-634f-4950-90dd-f7436f07098b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:33.481215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1282/2558] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:33.485326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd233eb-33d9-4f77-afed-a7d7e8212040 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:33.496337 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd233eb-33d9-4f77-afed-a7d7e8212040 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:33.501015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab9e8c-c967-4662-bee3-baada125bc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:33.501897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab9e8c-c967-4662-bee3-baada125bc49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:33.502184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab9e8c-c967-4662-bee3-baada125bc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:33.502462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab9e8c-c967-4662-bee3-baada125bc49 admin admin] PATH_INFO: `/v3/projects/121ea24896c74156b61335ef15a07e45/users/804a2984241f4d428f076e41b62bca07/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:33.504505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab9e8c-c967-4662-bee3-baada125bc49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121ea24896c74156b61335ef15a07e45, user_id=804a2984241f4d428f076e41b62bca07, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:33.507148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab9e8c-c967-4662-bee3-baada125bc49 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:33.517535 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1277/2559] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:12:33 2022] PUT /identity/v3/projects/121ea24896c74156b61335ef15a07e45/users/804a2984241f4d428f076e41b62bca07/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:33.527549 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a99c-9e04-48d7-8508-b4c06cf75b31 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:33.527846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a99c-9e04-48d7-8508-b4c06cf75b31 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:33.528134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758a99c-9e04-48d7-8508-b4c06cf75b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:33.554726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-6758a99c-9e04-48d7-8508-b4c06cf75b31 None None] MFA Rules not processed for user `804a2984241f4d428f076e41b62bca07`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:33.555950 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6758a99c-9e04-48d7-8508-b4c06cf75b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:33.574277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:33.631355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1283/2560] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:33 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:12:33.643614 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab9e8c-c967-4662-bee3-baada125bc49 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:33.645431 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-68ab9e8c-c967-4662-bee3-baada125bc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:33.668840 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:33.703992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab9e8c-c967-4662-bee3-baada125bc49 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:33.707928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c514cf-b269-4e9c-b5c2-523d81fa1a1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:33.708603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c514cf-b269-4e9c-b5c2-523d81fa1a1c service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:33.708888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c514cf-b269-4e9c-b5c2-523d81fa1a1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:33.709163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c514cf-b269-4e9c-b5c2-523d81fa1a1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:33.718779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c514cf-b269-4e9c-b5c2-523d81fa1a1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:33.720207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c514cf-b269-4e9c-b5c2-523d81fa1a1c service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:33.729899 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1278/2561] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:33 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:33.771440 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758a99c-9e04-48d7-8508-b4c06cf75b31 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:33.772969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6758a99c-9e04-48d7-8508-b4c06cf75b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:33.785382 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758a99c-9e04-48d7-8508-b4c06cf75b31 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:33.789104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e82a5-1e28-4ddf-831c-6a94b8911003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:33.789767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e82a5-1e28-4ddf-831c-6a94b8911003 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:33.790053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e82a5-1e28-4ddf-831c-6a94b8911003 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:33.790327 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e82a5-1e28-4ddf-831c-6a94b8911003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:33.846514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e82a5-1e28-4ddf-831c-6a94b8911003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:33.847994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e82a5-1e28-4ddf-831c-6a94b8911003 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:33.856836 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1284/2562] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:36.275053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c514cf-b269-4e9c-b5c2-523d81fa1a1c service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:36.276676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-40c514cf-b269-4e9c-b5c2-523d81fa1a1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:36.333136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c514cf-b269-4e9c-b5c2-523d81fa1a1c service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:36.336664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de85272-65e7-4d40-9768-108384ea1a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:36.337299 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de85272-65e7-4d40-9768-108384ea1a17 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:36.337581 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de85272-65e7-4d40-9768-108384ea1a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:36.337854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de85272-65e7-4d40-9768-108384ea1a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:36.339670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2de85272-65e7-4d40-9768-108384ea1a17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:36.402890 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de85272-65e7-4d40-9768-108384ea1a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:36.404331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de85272-65e7-4d40-9768-108384ea1a17 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:36.414065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1279/2563] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:37.149422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e82a5-1e28-4ddf-831c-6a94b8911003 service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:37.215656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e82a5-1e28-4ddf-831c-6a94b8911003 service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:37.220152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:37.221059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:37.221735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:37.222026 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] PATH_INFO: `/v3/users/6930e78c8a8b44c8a6c136b68f12c2a4` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:37.223279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6930e78c8a8b44c8a6c136b68f12c2a4)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:37.225406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:37.281123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6930e78c8a8b44c8a6c136b68f12c2a4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:37.281820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6930e78c8a8b44c8a6c136b68f12c2a4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:37.290027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6930e78c8a8b44c8a6c136b68f12c2a4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:37.290674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6930e78c8a8b44c8a6c136b68f12c2a4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:37.291947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:37.305098 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6930e78c8a8b44c8a6c136b68f12c2a4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:37.318780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1285/2564] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:12:37 2022] DELETE /identity/v3/users/6930e78c8a8b44c8a6c136b68f12c2a4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:37.324094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c24ca9-5fb4-4f83-b5ef-8e5c1b131b5a None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:37.324611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c24ca9-5fb4-4f83-b5ef-8e5c1b131b5a None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:37.324785 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c24ca9-5fb4-4f83-b5ef-8e5c1b131b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:37.341948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-f1c24ca9-5fb4-4f83-b5ef-8e5c1b131b5a None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:37.343292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f1c24ca9-5fb4-4f83-b5ef-8e5c1b131b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:37.406168 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1280/2565] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:37.420237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:37.424597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:37.483833 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4172d-ce1f-49e5-bdd8-4ef4c90a5f77 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:37.487769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b73cca-c5a1-4864-a586-4dd291c3bf49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:37.488349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b73cca-c5a1-4864-a586-4dd291c3bf49 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:37.488656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b73cca-c5a1-4864-a586-4dd291c3bf49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:37.488938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b73cca-c5a1-4864-a586-4dd291c3bf49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:37.499360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b73cca-c5a1-4864-a586-4dd291c3bf49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:37.501236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b73cca-c5a1-4864-a586-4dd291c3bf49 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:37.510846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1286/2566] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:37.943384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c24ca9-5fb4-4f83-b5ef-8e5c1b131b5a None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:37.944962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f1c24ca9-5fb4-4f83-b5ef-8e5c1b131b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:37.957694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c24ca9-5fb4-4f83-b5ef-8e5c1b131b5a None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:37.972143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:37.973496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:37.973913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:37.974368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] PATH_INFO: `/v3/projects/b23f47cea0cd48fea5346e9331c1da49` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:37.976025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23f47cea0cd48fea5346e9331c1da49)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:37.978090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:37.987917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:38.031082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23f47cea0cd48fea5346e9331c1da49', 'request_id': 'req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:38.037612 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23f47cea0cd48fea5346e9331c1da49', 'request_id': 'req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:38.047123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23f47cea0cd48fea5346e9331c1da49', 'request_id': 'req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:38.058832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] The token cache is being invalidate because project b23f47cea0cd48fea5346e9331c1da49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:12:38.059116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:38.059744 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:38.061315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1281/2567] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:12:37 2022] DELETE /identity/v3/projects/b23f47cea0cd48fea5346e9331c1da49 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:38.065408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b73cca-c5a1-4864-a586-4dd291c3bf49 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:38.075801 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b73cca-c5a1-4864-a586-4dd291c3bf49 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:38.147154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:38.148268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:38.148268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:38.148268 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] PATH_INFO: `/v3/users/804a2984241f4d428f076e41b62bca07` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:38.150253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=804a2984241f4d428f076e41b62bca07)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:38.153648 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:38.233287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '804a2984241f4d428f076e41b62bca07'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:38.236965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '804a2984241f4d428f076e41b62bca07'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:38.253670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '804a2984241f4d428f076e41b62bca07'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:38.254461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '804a2984241f4d428f076e41b62bca07'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:38.274515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16a4fca8-c731-4554-88af-26a45d152fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '804a2984241f4d428f076e41b62bca07'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:38.280376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:38.308524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1287/2568] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:12:38 2022] DELETE /identity/v3/users/804a2984241f4d428f076e41b62bca07 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:38.818786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:38.821053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:38.849220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba5e9c-1774-4b4c-a7ce-0a2e6819cc4d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:38.957814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:38.958694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:38.958980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:38.959263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] PATH_INFO: `/v3/projects/121ea24896c74156b61335ef15a07e45` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:38.960503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=121ea24896c74156b61335ef15a07e45)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:38.962457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:39.023351 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121ea24896c74156b61335ef15a07e45', 'request_id': 'req-e9ac5fc2-8999-4f05-b360-c08ac185e510'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:39.032670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121ea24896c74156b61335ef15a07e45', 'request_id': 'req-e9ac5fc2-8999-4f05-b360-c08ac185e510'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:39.044800 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121ea24896c74156b61335ef15a07e45', 'request_id': 'req-e9ac5fc2-8999-4f05-b360-c08ac185e510'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:39.057025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] The token cache is being invalidate because project 121ea24896c74156b61335ef15a07e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:12:39.057321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:39.059253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e9ac5fc2-8999-4f05-b360-c08ac185e510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:39.061160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1282/2569] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:12:38 2022] DELETE /identity/v3/projects/121ea24896c74156b61335ef15a07e45 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:39.065786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2278f-b370-4607-81f2-7cc19e189966 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.066082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2278f-b370-4607-81f2-7cc19e189966 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.066367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2278f-b370-4607-81f2-7cc19e189966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.087822 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-80e2278f-b370-4607-81f2-7cc19e189966 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:39.090008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-80e2278f-b370-4607-81f2-7cc19e189966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:39.174057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1288/2570] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:39.176784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359fcbf-e63a-4d88-a8d6-7b0c067e2ef6 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.177085 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359fcbf-e63a-4d88-a8d6-7b0c067e2ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.177364 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359fcbf-e63a-4d88-a8d6-7b0c067e2ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.198703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-3359fcbf-e63a-4d88-a8d6-7b0c067e2ef6 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:39.200132 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3359fcbf-e63a-4d88-a8d6-7b0c067e2ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:39.208342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1283/2571] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:39.213215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2278f-b370-4607-81f2-7cc19e189966 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:39.224266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2278f-b370-4607-81f2-7cc19e189966 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:39.232853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71781a-218d-4377-8cf4-b2b3e2bbd349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:39.233044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71781a-218d-4377-8cf4-b2b3e2bbd349 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.233288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71781a-218d-4377-8cf4-b2b3e2bbd349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.233575 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71781a-218d-4377-8cf4-b2b3e2bbd349 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.234318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71781a-218d-4377-8cf4-b2b3e2bbd349 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:12:39.234630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71781a-218d-4377-8cf4-b2b3e2bbd349 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:39.236528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71781a-218d-4377-8cf4-b2b3e2bbd349 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:39.254826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1289/2572] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:12:39.258240 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359fcbf-e63a-4d88-a8d6-7b0c067e2ef6 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:39.270373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359fcbf-e63a-4d88-a8d6-7b0c067e2ef6 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:39.275452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8eb7f9-0c45-42ee-9096-e94ada776ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:39.276284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8eb7f9-0c45-42ee-9096-e94ada776ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.277488 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8eb7f9-0c45-42ee-9096-e94ada776ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.278143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8eb7f9-0c45-42ee-9096-e94ada776ed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.278956 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8eb7f9-0c45-42ee-9096-e94ada776ed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:39.281057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8eb7f9-0c45-42ee-9096-e94ada776ed1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:39.305128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:39.318438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1284/2573] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:12:39 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:39.319640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71781a-218d-4377-8cf4-b2b3e2bbd349 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:39.334121 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71781a-218d-4377-8cf4-b2b3e2bbd349 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:39.413975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27592052-6eed-4695-818a-1c5630471060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:39.414580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27592052-6eed-4695-818a-1c5630471060 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.414989 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27592052-6eed-4695-818a-1c5630471060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.415649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27592052-6eed-4695-818a-1c5630471060 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.416372 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27592052-6eed-4695-818a-1c5630471060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:39.419448 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27592052-6eed-4695-818a-1c5630471060 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:39.442145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1290/2574] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:12:39 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:39.446078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8eb7f9-0c45-42ee-9096-e94ada776ed1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:39.457305 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8eb7f9-0c45-42ee-9096-e94ada776ed1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:39.462117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc6d79-512f-4038-b35e-d64646226741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:39.462897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc6d79-512f-4038-b35e-d64646226741 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.463252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc6d79-512f-4038-b35e-d64646226741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.463603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc6d79-512f-4038-b35e-d64646226741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.464774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc6d79-512f-4038-b35e-d64646226741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:39.466463 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc6d79-512f-4038-b35e-d64646226741 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:39.477783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1285/2575] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:39.482171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27592052-6eed-4695-818a-1c5630471060 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:39.497222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27592052-6eed-4695-818a-1c5630471060 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:39.507203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac04d6-9194-46a6-899c-f63d4389621a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:39.508215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac04d6-9194-46a6-899c-f63d4389621a admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.508703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac04d6-9194-46a6-899c-f63d4389621a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.509022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac04d6-9194-46a6-899c-f63d4389621a admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.510043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac04d6-9194-46a6-899c-f63d4389621a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:39.512898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac04d6-9194-46a6-899c-f63d4389621a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:39.524153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1291/2576] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:39.529426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc6d79-512f-4038-b35e-d64646226741 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:39.542691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc6d79-512f-4038-b35e-d64646226741 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:39.547417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a77d-27ba-4352-9973-3a59f550171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:39.548303 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a77d-27ba-4352-9973-3a59f550171c admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.548629 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a77d-27ba-4352-9973-3a59f550171c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.548907 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a77d-27ba-4352-9973-3a59f550171c admin admin] PATH_INFO: `/v3/projects/b57fc27a63df4ed7a707fde362712219/users/04688cf01e344193a59f956895d6ca8f/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.564246 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61a77d-27ba-4352-9973-3a59f550171c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b57fc27a63df4ed7a707fde362712219, user_id=04688cf01e344193a59f956895d6ca8f, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:39.566972 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61a77d-27ba-4352-9973-3a59f550171c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:39.581600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1286/2577] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:12:39 2022] PUT /identity/v3/projects/b57fc27a63df4ed7a707fde362712219/users/04688cf01e344193a59f956895d6ca8f/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:39.592373 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071c570-a9b2-4fd6-bd16-e9972d1a3120 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.592703 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071c570-a9b2-4fd6-bd16-e9972d1a3120 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.592991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071c570-a9b2-4fd6-bd16-e9972d1a3120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.621524 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-3071c570-a9b2-4fd6-bd16-e9972d1a3120 None None] MFA Rules not processed for user `04688cf01e344193a59f956895d6ca8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:39.622747 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-3071c570-a9b2-4fd6-bd16-e9972d1a3120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:39.678082 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:39.696680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1292/2578] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:39.701725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a77d-27ba-4352-9973-3a59f550171c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:39.763584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a77d-27ba-4352-9973-3a59f550171c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:39.768298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d1784-71f9-4931-a2d0-1cf17e805a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:39.768991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d1784-71f9-4931-a2d0-1cf17e805a79 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.769295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d1784-71f9-4931-a2d0-1cf17e805a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.769551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d1784-71f9-4931-a2d0-1cf17e805a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.770286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d1784-71f9-4931-a2d0-1cf17e805a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:39.772266 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d1784-71f9-4931-a2d0-1cf17e805a79 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:39.802502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1287/2579] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:12:39 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:39.804006 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071c570-a9b2-4fd6-bd16-e9972d1a3120 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:39.815328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071c570-a9b2-4fd6-bd16-e9972d1a3120 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:39.875297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1d5e0-6873-4b66-b6c1-ab6bba721d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:39.875895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1d5e0-6873-4b66-b6c1-ab6bba721d83 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.876181 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1d5e0-6873-4b66-b6c1-ab6bba721d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.876496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1d5e0-6873-4b66-b6c1-ab6bba721d83 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.877209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1d5e0-6873-4b66-b6c1-ab6bba721d83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:39.879177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1d5e0-6873-4b66-b6c1-ab6bba721d83 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:39.902259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1293/2580] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:12:39 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:39.906764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d1784-71f9-4931-a2d0-1cf17e805a79 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:39.922804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d1784-71f9-4931-a2d0-1cf17e805a79 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:39.927650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32ac85-764a-4b03-85d8-19dd294c1e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:39.928457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ac85-764a-4b03-85d8-19dd294c1e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.928779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ac85-764a-4b03-85d8-19dd294c1e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.929083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ac85-764a-4b03-85d8-19dd294c1e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.929879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32ac85-764a-4b03-85d8-19dd294c1e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:39.931786 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32ac85-764a-4b03-85d8-19dd294c1e79 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:39.948485 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1288/2581] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:39.949730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1d5e0-6873-4b66-b6c1-ab6bba721d83 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:39.962706 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1d5e0-6873-4b66-b6c1-ab6bba721d83 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:39.969272 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0382ee-78e4-4e36-ad7d-90c69c331390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:39.970188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0382ee-78e4-4e36-ad7d-90c69c331390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:39.970477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0382ee-78e4-4e36-ad7d-90c69c331390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:39.970817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0382ee-78e4-4e36-ad7d-90c69c331390 admin admin] PATH_INFO: `/v3/projects/6a07de1029914b3981a47d9064a36e97/users/e2a523a76a214c10a54e1eec80626b94/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:39.988732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0382ee-78e4-4e36-ad7d-90c69c331390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a07de1029914b3981a47d9064a36e97, user_id=e2a523a76a214c10a54e1eec80626b94, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:39.991439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0382ee-78e4-4e36-ad7d-90c69c331390 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:40.023374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1294/2582] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:12:39 2022] PUT /identity/v3/projects/6a07de1029914b3981a47d9064a36e97/users/e2a523a76a214c10a54e1eec80626b94/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:40.028280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32ac85-764a-4b03-85d8-19dd294c1e79 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:40.042461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32ac85-764a-4b03-85d8-19dd294c1e79 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:40.128010 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4a6dc-01db-4e1c-b76b-1940da1c692e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:40.128908 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4a6dc-01db-4e1c-b76b-1940da1c692e admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:40.129219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4a6dc-01db-4e1c-b76b-1940da1c692e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:40.129494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4a6dc-01db-4e1c-b76b-1940da1c692e admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:40.130226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4a6dc-01db-4e1c-b76b-1940da1c692e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:40.131995 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4a6dc-01db-4e1c-b76b-1940da1c692e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:40.150784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1289/2583] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:40.156385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0382ee-78e4-4e36-ad7d-90c69c331390 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:40.171203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0382ee-78e4-4e36-ad7d-90c69c331390 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:40.178090 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec5390-6a1e-4c1e-ae42-5f944ef739ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:40.180492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec5390-6a1e-4c1e-ae42-5f944ef739ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:40.180674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec5390-6a1e-4c1e-ae42-5f944ef739ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:40.180956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec5390-6a1e-4c1e-ae42-5f944ef739ac admin admin] PATH_INFO: `/v3/projects/6a07de1029914b3981a47d9064a36e97/users/e2a523a76a214c10a54e1eec80626b94/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:40.184016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec5390-6a1e-4c1e-ae42-5f944ef739ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a07de1029914b3981a47d9064a36e97, user_id=e2a523a76a214c10a54e1eec80626b94, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:40.186848 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec5390-6a1e-4c1e-ae42-5f944ef739ac admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:40.201598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1295/2584] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:12:40 2022] PUT /identity/v3/projects/6a07de1029914b3981a47d9064a36e97/users/e2a523a76a214c10a54e1eec80626b94/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:40.212806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61679d04-d454-4a10-b988-3afdaaff7b6c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:40.213081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61679d04-d454-4a10-b988-3afdaaff7b6c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:40.213360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61679d04-d454-4a10-b988-3afdaaff7b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:40.247927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-61679d04-d454-4a10-b988-3afdaaff7b6c None None] MFA Rules not processed for user `e2a523a76a214c10a54e1eec80626b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:40.249318 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-61679d04-d454-4a10-b988-3afdaaff7b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:40.276064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:40.346496 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1290/2585] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:40.357027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec5390-6a1e-4c1e-ae42-5f944ef739ac admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:40.359141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-71ec5390-6a1e-4c1e-ae42-5f944ef739ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:40.436627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec5390-6a1e-4c1e-ae42-5f944ef739ac admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:40.441074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41e57b-7022-4680-a465-78e51f5b1c88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:40.441809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41e57b-7022-4680-a465-78e51f5b1c88 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:40.442240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41e57b-7022-4680-a465-78e51f5b1c88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:40.442527 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41e57b-7022-4680-a465-78e51f5b1c88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:40.454142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41e57b-7022-4680-a465-78e51f5b1c88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:40.455735 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41e57b-7022-4680-a465-78e51f5b1c88 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:40.466087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1296/2586] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:40.809635 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61679d04-d454-4a10-b988-3afdaaff7b6c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:40.819055 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61679d04-d454-4a10-b988-3afdaaff7b6c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:40.824293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a0dd9-3ebf-4a64-82d3-f412ba7b763a tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2a523a76a214c10a54e1eec80626b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a07de1029914b3981a47d9064a36e97', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:40.824994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a0dd9-3ebf-4a64-82d3-f412ba7b763a tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:40.825284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a0dd9-3ebf-4a64-82d3-f412ba7b763a tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:40.825566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a0dd9-3ebf-4a64-82d3-f412ba7b763a tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:40.826295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a0dd9-3ebf-4a64-82d3-f412ba7b763a tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:12:40.826607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a0dd9-3ebf-4a64-82d3-f412ba7b763a tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:40.828252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a0dd9-3ebf-4a64-82d3-f412ba7b763a tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:40.841201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1291/2587] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:12:40.845344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41e57b-7022-4680-a465-78e51f5b1c88 service cinder] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:40.854594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41e57b-7022-4680-a465-78e51f5b1c88 service cinder] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:40.859419 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be883f-63fb-4029-b9b4-65b0f9c4baf3 tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2a523a76a214c10a54e1eec80626b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a07de1029914b3981a47d9064a36e97', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:40.860076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be883f-63fb-4029-b9b4-65b0f9c4baf3 tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:40.860361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be883f-63fb-4029-b9b4-65b0f9c4baf3 tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:40.860671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be883f-63fb-4029-b9b4-65b0f9c4baf3 tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:40.861369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be883f-63fb-4029-b9b4-65b0f9c4baf3 tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:40.863287 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be883f-63fb-4029-b9b4-65b0f9c4baf3 tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:40.890513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1297/2588] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:12:40 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:40.894596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a0dd9-3ebf-4a64-82d3-f412ba7b763a tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:40.905111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a0dd9-3ebf-4a64-82d3-f412ba7b763a tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:40.961103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154797c6-96b2-4e5c-bffa-14412f072c9b tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2a523a76a214c10a54e1eec80626b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a07de1029914b3981a47d9064a36e97', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:40.961818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154797c6-96b2-4e5c-bffa-14412f072c9b tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:40.961970 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154797c6-96b2-4e5c-bffa-14412f072c9b tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:40.962249 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154797c6-96b2-4e5c-bffa-14412f072c9b tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:40.963020 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154797c6-96b2-4e5c-bffa-14412f072c9b tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:12:40.963333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154797c6-96b2-4e5c-bffa-14412f072c9b tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:40.965320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154797c6-96b2-4e5c-bffa-14412f072c9b tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:40.976545 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1292/2589] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:12:41.012519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be883f-63fb-4029-b9b4-65b0f9c4baf3 tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:41.022751 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be883f-63fb-4029-b9b4-65b0f9c4baf3 tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:41.027998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2a523a76a214c10a54e1eec80626b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a07de1029914b3981a47d9064a36e97', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:41.028887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:41.029175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:41.029451 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] PATH_INFO: `/v3/projects/e41421c1f1864d5d80c4c8438ddbe702` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:41.030660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e41421c1f1864d5d80c4c8438ddbe702)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:41.032644 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:41.053967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:41.079280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41421c1f1864d5d80c4c8438ddbe702', 'request_id': 'req-a54fe725-936c-4e39-a0dd-b96110eb034e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:41.084783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41421c1f1864d5d80c4c8438ddbe702', 'request_id': 'req-a54fe725-936c-4e39-a0dd-b96110eb034e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:41.091558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41421c1f1864d5d80c4c8438ddbe702', 'request_id': 'req-a54fe725-936c-4e39-a0dd-b96110eb034e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:41.103338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] The token cache is being invalidate because project e41421c1f1864d5d80c4c8438ddbe702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:12:41.103630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:41.104269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-a54fe725-936c-4e39-a0dd-b96110eb034e tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:41.105854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1298/2590] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:12:41 2022] DELETE /identity/v3/projects/e41421c1f1864d5d80c4c8438ddbe702 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:41.293140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154797c6-96b2-4e5c-bffa-14412f072c9b tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:41.294855 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-154797c6-96b2-4e5c-bffa-14412f072c9b tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:41.359116 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154797c6-96b2-4e5c-bffa-14412f072c9b tempest-VolumeQuotaClassesTest-411750566 tempest-VolumeQuotaClassesTest-411750566-project-admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:41.403397 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:41.416766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:41.418242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:41.418536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:41.418809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] PATH_INFO: `/v3/users/04688cf01e344193a59f956895d6ca8f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:41.420021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04688cf01e344193a59f956895d6ca8f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:41.422031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:41.467007 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04688cf01e344193a59f956895d6ca8f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:41.467702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04688cf01e344193a59f956895d6ca8f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:41.474398 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04688cf01e344193a59f956895d6ca8f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:41.475013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04688cf01e344193a59f956895d6ca8f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:41.482186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04688cf01e344193a59f956895d6ca8f'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:41.499817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1293/2591] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:12:41 2022] DELETE /identity/v3/users/04688cf01e344193a59f956895d6ca8f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:41.504295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633901fa-8264-4dfa-a4c7-7bfb5ea79d61 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:41.504800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633901fa-8264-4dfa-a4c7-7bfb5ea79d61 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:41.504916 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633901fa-8264-4dfa-a4c7-7bfb5ea79d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:41.522975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-633901fa-8264-4dfa-a4c7-7bfb5ea79d61 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:41.524167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-633901fa-8264-4dfa-a4c7-7bfb5ea79d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:41.583892 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1299/2592] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:41.593491 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:41.594932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:41.651059 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b47160-5de5-4e7f-a8b0-d248352e5c33 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:41.654757 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808e55f-63aa-473e-a439-a82b974144c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:41.655400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808e55f-63aa-473e-a439-a82b974144c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:41.655682 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808e55f-63aa-473e-a439-a82b974144c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:41.655955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808e55f-63aa-473e-a439-a82b974144c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:41.665131 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808e55f-63aa-473e-a439-a82b974144c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:41.666563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808e55f-63aa-473e-a439-a82b974144c9 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:41.676699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1294/2593] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:41.940553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633901fa-8264-4dfa-a4c7-7bfb5ea79d61 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:41.942150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-633901fa-8264-4dfa-a4c7-7bfb5ea79d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:41.955152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633901fa-8264-4dfa-a4c7-7bfb5ea79d61 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:41.960228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:41.961240 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:41.961579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:41.961946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] PATH_INFO: `/v3/projects/b57fc27a63df4ed7a707fde362712219` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:41.963191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b57fc27a63df4ed7a707fde362712219)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:41.965740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:42.019592 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b57fc27a63df4ed7a707fde362712219', 'request_id': 'req-85497acf-6e18-4af0-b956-96fb58546266'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.025180 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b57fc27a63df4ed7a707fde362712219', 'request_id': 'req-85497acf-6e18-4af0-b956-96fb58546266'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.031704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b57fc27a63df4ed7a707fde362712219', 'request_id': 'req-85497acf-6e18-4af0-b956-96fb58546266'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.045022 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] The token cache is being invalidate because project b57fc27a63df4ed7a707fde362712219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:12:42.045311 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.045937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.047486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1300/2594] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:12:41 2022] DELETE /identity/v3/projects/b57fc27a63df4ed7a707fde362712219 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:42.051505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808e55f-63aa-473e-a439-a82b974144c9 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:42.061820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808e55f-63aa-473e-a439-a82b974144c9 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:42.117513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:42.118384 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:42.118671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:42.118947 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] PATH_INFO: `/v3/users/e2a523a76a214c10a54e1eec80626b94` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:42.120817 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a523a76a214c10a54e1eec80626b94)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:42.122766 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:42.132872 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:42.169936 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a523a76a214c10a54e1eec80626b94'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.170606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a523a76a214c10a54e1eec80626b94'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.177674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a523a76a214c10a54e1eec80626b94'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.178315 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a523a76a214c10a54e1eec80626b94'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.185957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-30c121d3-649d-4a07-a43a-5a19ecab0ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a523a76a214c10a54e1eec80626b94'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.204112 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1295/2595] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:12:42 2022] DELETE /identity/v3/users/e2a523a76a214c10a54e1eec80626b94 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:42.477199 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:42.478640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:42.496304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85497acf-6e18-4af0-b956-96fb58546266 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:42.526748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:42.556770 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:42.557568 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:42.558529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:42.558812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] PATH_INFO: `/v3/projects/6a07de1029914b3981a47d9064a36e97` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:42.560050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a07de1029914b3981a47d9064a36e97)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:42.562075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:42.614990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a07de1029914b3981a47d9064a36e97', 'request_id': 'req-7287f27c-87f0-4ba6-8c14-643662c22daf'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.622401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a07de1029914b3981a47d9064a36e97', 'request_id': 'req-7287f27c-87f0-4ba6-8c14-643662c22daf'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.628624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a07de1029914b3981a47d9064a36e97', 'request_id': 'req-7287f27c-87f0-4ba6-8c14-643662c22daf'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.642253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] The token cache is being invalidate because project 6a07de1029914b3981a47d9064a36e97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:12:42.642538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.643159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-7287f27c-87f0-4ba6-8c14-643662c22daf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:42.644736 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1301/2596] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:12:42 2022] DELETE /identity/v3/projects/6a07de1029914b3981a47d9064a36e97 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:12:42.649725 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b5edb-dd6d-4e96-8ecf-762b3f3a7d7c None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:42.650086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b5edb-dd6d-4e96-8ecf-762b3f3a7d7c None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:42.650434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b5edb-dd6d-4e96-8ecf-762b3f3a7d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:42.669363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-6b4b5edb-dd6d-4e96-8ecf-762b3f3a7d7c None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:42.670574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6b4b5edb-dd6d-4e96-8ecf-762b3f3a7d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:42.728196 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1296/2597] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:42.736547 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9d673-b9d8-4ddd-b6d3-9b47e8fe791e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:42.736846 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9d673-b9d8-4ddd-b6d3-9b47e8fe791e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:42.737127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9d673-b9d8-4ddd-b6d3-9b47e8fe791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:42.755072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-76d9d673-b9d8-4ddd-b6d3-9b47e8fe791e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:42.756264 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-76d9d673-b9d8-4ddd-b6d3-9b47e8fe791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:42.764349 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1302/2598] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:42.768374 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b5edb-dd6d-4e96-8ecf-762b3f3a7d7c None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:42.779307 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b5edb-dd6d-4e96-8ecf-762b3f3a7d7c None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:42.784049 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfffecb-b260-47b6-bbae-8c5ebe2489be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:42.784731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfffecb-b260-47b6-bbae-8c5ebe2489be admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:42.785013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfffecb-b260-47b6-bbae-8c5ebe2489be admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:42.785286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfffecb-b260-47b6-bbae-8c5ebe2489be admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:42.785996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfffecb-b260-47b6-bbae-8c5ebe2489be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:12:42.786299 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfffecb-b260-47b6-bbae-8c5ebe2489be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:42.787895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfffecb-b260-47b6-bbae-8c5ebe2489be admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:42.800285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1297/2599] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:12:42.804434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9d673-b9d8-4ddd-b6d3-9b47e8fe791e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:42.813859 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9d673-b9d8-4ddd-b6d3-9b47e8fe791e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:42.818431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635934-8d6d-41ec-8c16-e31379e76016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:42.819065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635934-8d6d-41ec-8c16-e31379e76016 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:42.819346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635934-8d6d-41ec-8c16-e31379e76016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:42.819619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635934-8d6d-41ec-8c16-e31379e76016 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:42.820310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13635934-8d6d-41ec-8c16-e31379e76016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:42.822343 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13635934-8d6d-41ec-8c16-e31379e76016 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:42.847473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1303/2600] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:12:42 2022] POST /identity/v3/projects => generated 399 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:42.851492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfffecb-b260-47b6-bbae-8c5ebe2489be admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:42.862681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfffecb-b260-47b6-bbae-8c5ebe2489be admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:42.921399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c8f80-cda6-42e8-be33-67379f519cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:42.922082 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c8f80-cda6-42e8-be33-67379f519cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:42.922363 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c8f80-cda6-42e8-be33-67379f519cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:42.922637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c8f80-cda6-42e8-be33-67379f519cdd admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:42.923328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c8f80-cda6-42e8-be33-67379f519cdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:42.925296 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c8f80-cda6-42e8-be33-67379f519cdd admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:42.945462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1298/2601] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:12:42 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:42.949533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635934-8d6d-41ec-8c16-e31379e76016 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:42.960001 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635934-8d6d-41ec-8c16-e31379e76016 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:42.965091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a65ec-ff06-4432-83df-d598b2349fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:42.965909 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a65ec-ff06-4432-83df-d598b2349fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:42.966191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a65ec-ff06-4432-83df-d598b2349fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:42.966464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a65ec-ff06-4432-83df-d598b2349fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:42.967184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a65ec-ff06-4432-83df-d598b2349fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:42.968812 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a65ec-ff06-4432-83df-d598b2349fcc admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:42.981108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1304/2602] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:42.985105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c8f80-cda6-42e8-be33-67379f519cdd admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:42.991383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:42.995606 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c8f80-cda6-42e8-be33-67379f519cdd admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:43.001484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d47a78-bccb-4c1f-9384-d237b507a98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:43.002357 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d47a78-bccb-4c1f-9384-d237b507a98b admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.002820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d47a78-bccb-4c1f-9384-d237b507a98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.003306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d47a78-bccb-4c1f-9384-d237b507a98b admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.004063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d47a78-bccb-4c1f-9384-d237b507a98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:43.005723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d47a78-bccb-4c1f-9384-d237b507a98b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:43.016365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1299/2603] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:43.020273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a65ec-ff06-4432-83df-d598b2349fcc admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:43.031519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a65ec-ff06-4432-83df-d598b2349fcc admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:43.036333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78212d49-6d93-40c1-86da-8f1685fcae80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:43.037231 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78212d49-6d93-40c1-86da-8f1685fcae80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.037511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78212d49-6d93-40c1-86da-8f1685fcae80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.037781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78212d49-6d93-40c1-86da-8f1685fcae80 admin admin] PATH_INFO: `/v3/projects/7948be7c1b6640bfb0d761cc94585624/users/7d81890002e842a9bae96d6881323166/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.052447 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78212d49-6d93-40c1-86da-8f1685fcae80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7948be7c1b6640bfb0d761cc94585624, user_id=7d81890002e842a9bae96d6881323166, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:43.055087 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78212d49-6d93-40c1-86da-8f1685fcae80 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:43.067839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1305/2604] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:12:43 2022] PUT /identity/v3/projects/7948be7c1b6640bfb0d761cc94585624/users/7d81890002e842a9bae96d6881323166/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:43.078613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112941-6b11-413d-955c-d18202e9158f None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.079075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112941-6b11-413d-955c-d18202e9158f None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.079170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112941-6b11-413d-955c-d18202e9158f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.106302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ee112941-6b11-413d-955c-d18202e9158f None None] MFA Rules not processed for user `7d81890002e842a9bae96d6881323166`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:43.107494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ee112941-6b11-413d-955c-d18202e9158f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:43.171632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1300/2605] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:43.176164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78212d49-6d93-40c1-86da-8f1685fcae80 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:43.238295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:43.242486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78212d49-6d93-40c1-86da-8f1685fcae80 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:43.249378 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ecd0e-6cf2-4f5b-9c42-483c684ef884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:43.250221 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ecd0e-6cf2-4f5b-9c42-483c684ef884 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.250503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ecd0e-6cf2-4f5b-9c42-483c684ef884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.250781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ecd0e-6cf2-4f5b-9c42-483c684ef884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.251468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ecd0e-6cf2-4f5b-9c42-483c684ef884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:43.253438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ecd0e-6cf2-4f5b-9c42-483c684ef884 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:43.279721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1306/2606] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:12:43 2022] POST /identity/v3/projects => generated 399 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:12:43.284130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee112941-6b11-413d-955c-d18202e9158f None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:43.295056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee112941-6b11-413d-955c-d18202e9158f None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:43.352429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eb282-e28d-4c16-97b1-ed290d2c71af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:43.353320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb282-e28d-4c16-97b1-ed290d2c71af admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.353320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb282-e28d-4c16-97b1-ed290d2c71af admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.353617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb282-e28d-4c16-97b1-ed290d2c71af admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.354834 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828eb282-e28d-4c16-97b1-ed290d2c71af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:43.356746 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828eb282-e28d-4c16-97b1-ed290d2c71af admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:43.379617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1301/2607] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:12:43 2022] POST /identity/v3/users => generated 306 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:43.380876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ecd0e-6cf2-4f5b-9c42-483c684ef884 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:43.392073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ecd0e-6cf2-4f5b-9c42-483c684ef884 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:43.396598 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd0ed1-57e7-497f-b17c-617b0f2ef939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:43.397249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0ed1-57e7-497f-b17c-617b0f2ef939 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.397533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0ed1-57e7-497f-b17c-617b0f2ef939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.397809 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0ed1-57e7-497f-b17c-617b0f2ef939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.398552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd0ed1-57e7-497f-b17c-617b0f2ef939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:43.400158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd0ed1-57e7-497f-b17c-617b0f2ef939 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:43.411742 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1307/2608] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:12:43.415728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eb282-e28d-4c16-97b1-ed290d2c71af admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:43.425895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eb282-e28d-4c16-97b1-ed290d2c71af admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:43.430519 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad41253-3078-4e2a-bdac-a61a95d57f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:43.431394 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad41253-3078-4e2a-bdac-a61a95d57f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.431673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad41253-3078-4e2a-bdac-a61a95d57f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.431946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad41253-3078-4e2a-bdac-a61a95d57f2d admin admin] PATH_INFO: `/v3/projects/3f018f848da04e3db1acbab748956153/users/0828ce43f6ad4ea393c5bf1f67014c43/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.447674 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad41253-3078-4e2a-bdac-a61a95d57f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f018f848da04e3db1acbab748956153, user_id=0828ce43f6ad4ea393c5bf1f67014c43, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:43.450484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad41253-3078-4e2a-bdac-a61a95d57f2d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:43.463079 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1302/2609] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:12:43 2022] PUT /identity/v3/projects/3f018f848da04e3db1acbab748956153/users/0828ce43f6ad4ea393c5bf1f67014c43/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:43.467631 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd0ed1-57e7-497f-b17c-617b0f2ef939 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:43.477554 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd0ed1-57e7-497f-b17c-617b0f2ef939 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:43.540620 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9770cf-f63e-470b-8626-58d3c2ed104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:43.542011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9770cf-f63e-470b-8626-58d3c2ed104f admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.542258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9770cf-f63e-470b-8626-58d3c2ed104f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.543589 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9770cf-f63e-470b-8626-58d3c2ed104f admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.544509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9770cf-f63e-470b-8626-58d3c2ed104f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:43.546494 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9770cf-f63e-470b-8626-58d3c2ed104f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:43.567913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1308/2610] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:43.574462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad41253-3078-4e2a-bdac-a61a95d57f2d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:43.592282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad41253-3078-4e2a-bdac-a61a95d57f2d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:43.597120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc1c42-351e-47dc-8233-4741255f1d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:43.598013 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1c42-351e-47dc-8233-4741255f1d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.598292 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1c42-351e-47dc-8233-4741255f1d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.598565 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1c42-351e-47dc-8233-4741255f1d4e admin admin] PATH_INFO: `/v3/projects/3f018f848da04e3db1acbab748956153/users/0828ce43f6ad4ea393c5bf1f67014c43/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.600532 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc1c42-351e-47dc-8233-4741255f1d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f018f848da04e3db1acbab748956153, user_id=0828ce43f6ad4ea393c5bf1f67014c43, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:43.603216 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc1c42-351e-47dc-8233-4741255f1d4e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:43.614513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1303/2611] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:12:43 2022] PUT /identity/v3/projects/3f018f848da04e3db1acbab748956153/users/0828ce43f6ad4ea393c5bf1f67014c43/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:43.626204 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f188ec-644c-4b31-a34b-8d74408a71a1 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.626204 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f188ec-644c-4b31-a34b-8d74408a71a1 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.626503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f188ec-644c-4b31-a34b-8d74408a71a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.654455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-25f188ec-644c-4b31-a34b-8d74408a71a1 None None] MFA Rules not processed for user `0828ce43f6ad4ea393c5bf1f67014c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:43.655688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-25f188ec-644c-4b31-a34b-8d74408a71a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:43.722456 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1309/2612] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:43 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:43.733536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc1c42-351e-47dc-8233-4741255f1d4e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:43.735403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-dfcc1c42-351e-47dc-8233-4741255f1d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:43.775904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:43.797513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc1c42-351e-47dc-8233-4741255f1d4e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:43.801122 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8e021-068d-4cad-be24-0b974f43ced8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:43.801763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8e021-068d-4cad-be24-0b974f43ced8 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.802041 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8e021-068d-4cad-be24-0b974f43ced8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.802311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8e021-068d-4cad-be24-0b974f43ced8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.811173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8e021-068d-4cad-be24-0b974f43ced8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:43.812618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8e021-068d-4cad-be24-0b974f43ced8 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:43.821830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1304/2613] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:43 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:43.898507 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f188ec-644c-4b31-a34b-8d74408a71a1 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:43.912075 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f188ec-644c-4b31-a34b-8d74408a71a1 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:43.915772 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabbe32-5775-43e0-88fe-30e36ae7e52a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:43.916429 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabbe32-5775-43e0-88fe-30e36ae7e52a service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:43.916721 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabbe32-5775-43e0-88fe-30e36ae7e52a service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:43.916998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabbe32-5775-43e0-88fe-30e36ae7e52a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:43.967321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:43.977246 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabbe32-5775-43e0-88fe-30e36ae7e52a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:43.978712 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabbe32-5775-43e0-88fe-30e36ae7e52a service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:43.987791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1310/2614] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:48.025103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8e021-068d-4cad-be24-0b974f43ced8 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:48.087206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8e021-068d-4cad-be24-0b974f43ced8 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:48.091825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:48.092720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:48.093002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:48.093277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] PATH_INFO: `/v3/users/7d81890002e842a9bae96d6881323166` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:48.094457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d81890002e842a9bae96d6881323166)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:48.096406 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:48.142382 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d81890002e842a9bae96d6881323166'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.142959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d81890002e842a9bae96d6881323166'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.149774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d81890002e842a9bae96d6881323166'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.150400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d81890002e842a9bae96d6881323166'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.156787 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d81890002e842a9bae96d6881323166'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.174008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1305/2615] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:12:48 2022] DELETE /identity/v3/users/7d81890002e842a9bae96d6881323166 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:48.178455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7cb9c-bb0c-40e3-ac19-8c846fac59c7 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:48.178947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7cb9c-bb0c-40e3-ac19-8c846fac59c7 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:48.179054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7cb9c-bb0c-40e3-ac19-8c846fac59c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:48.196551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-99e7cb9c-bb0c-40e3-ac19-8c846fac59c7 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:48.197758 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-99e7cb9c-bb0c-40e3-ac19-8c846fac59c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:48.256198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1311/2616] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:48.265806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:48.267226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:48.323731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adf30d-b1ea-4d9d-952b-c5b8767b6836 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:48.327399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766142dc-38e2-420a-960f-99032719c2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:48.328037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766142dc-38e2-420a-960f-99032719c2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:48.328336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766142dc-38e2-420a-960f-99032719c2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:48.328618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766142dc-38e2-420a-960f-99032719c2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:48.337477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766142dc-38e2-420a-960f-99032719c2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:48.339006 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766142dc-38e2-420a-960f-99032719c2c2 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:48.344100 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:48.349285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1306/2617] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:48.612895 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7cb9c-bb0c-40e3-ac19-8c846fac59c7 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:48.614385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-99e7cb9c-bb0c-40e3-ac19-8c846fac59c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:48.627941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7cb9c-bb0c-40e3-ac19-8c846fac59c7 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:48.632634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:48.633509 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:48.633817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:48.634095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] PATH_INFO: `/v3/projects/7948be7c1b6640bfb0d761cc94585624` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:48.635275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7948be7c1b6640bfb0d761cc94585624)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:48.637500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:48.685432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7948be7c1b6640bfb0d761cc94585624', 'request_id': 'req-954281dc-2247-497c-a817-e188cb9ce900'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.691004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7948be7c1b6640bfb0d761cc94585624', 'request_id': 'req-954281dc-2247-497c-a817-e188cb9ce900'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.697059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7948be7c1b6640bfb0d761cc94585624', 'request_id': 'req-954281dc-2247-497c-a817-e188cb9ce900'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.709437 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] The token cache is being invalidate because project 7948be7c1b6640bfb0d761cc94585624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:12:48.709726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.710368 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.711885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1312/2618] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:12:48 2022] DELETE /identity/v3/projects/7948be7c1b6640bfb0d761cc94585624 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:48.716754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766142dc-38e2-420a-960f-99032719c2c2 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:48.726340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766142dc-38e2-420a-960f-99032719c2c2 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:48.785403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:48.786280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:48.786566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:48.786843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] PATH_INFO: `/v3/users/0828ce43f6ad4ea393c5bf1f67014c43` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:48.788025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0828ce43f6ad4ea393c5bf1f67014c43)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:48.790014 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:48.835647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0828ce43f6ad4ea393c5bf1f67014c43'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.836284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0828ce43f6ad4ea393c5bf1f67014c43'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.844324 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0828ce43f6ad4ea393c5bf1f67014c43'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.844987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0828ce43f6ad4ea393c5bf1f67014c43'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.852600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-40d8028d-687a-4763-bb0b-efad0e1236bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0828ce43f6ad4ea393c5bf1f67014c43'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:48.870923 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1307/2619] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:12:48 2022] DELETE /identity/v3/users/0828ce43f6ad4ea393c5bf1f67014c43 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:49.140078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:49.141588 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:49.155705 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954281dc-2247-497c-a817-e188cb9ce900 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:49.210997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:49.211780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.212061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.212333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] PATH_INFO: `/v3/projects/3f018f848da04e3db1acbab748956153` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.213525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f018f848da04e3db1acbab748956153)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:49.215504 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:49.232099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:49.262842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f018f848da04e3db1acbab748956153', 'request_id': 'req-d9943b7d-a9cb-4e9c-9b91-1300febac142'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:49.268242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f018f848da04e3db1acbab748956153', 'request_id': 'req-d9943b7d-a9cb-4e9c-9b91-1300febac142'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:49.274286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f018f848da04e3db1acbab748956153', 'request_id': 'req-d9943b7d-a9cb-4e9c-9b91-1300febac142'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:49.286984 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] The token cache is being invalidate because project 3f018f848da04e3db1acbab748956153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:12:49.287279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:49.287932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d9943b7d-a9cb-4e9c-9b91-1300febac142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:12:49.289530 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1313/2620] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:12:49 2022] DELETE /identity/v3/projects/3f018f848da04e3db1acbab748956153 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:12:49.295071 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4d6eb-671a-453d-9ce4-82be40e931ff None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.295538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4d6eb-671a-453d-9ce4-82be40e931ff None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.295633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4d6eb-671a-453d-9ce4-82be40e931ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.313858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-31c4d6eb-671a-453d-9ce4-82be40e931ff None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:49.315051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-31c4d6eb-671a-453d-9ce4-82be40e931ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:49.374652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1308/2621] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:49.382477 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352862f-3a7e-47ac-bcea-528d88604bea None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.382775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352862f-3a7e-47ac-bcea-528d88604bea None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.383088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352862f-3a7e-47ac-bcea-528d88604bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.402371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-0352862f-3a7e-47ac-bcea-528d88604bea None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:49.403657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0352862f-3a7e-47ac-bcea-528d88604bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:49.411775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1314/2622] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:12:49.416035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4d6eb-671a-453d-9ce4-82be40e931ff None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:49.426207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4d6eb-671a-453d-9ce4-82be40e931ff None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:49.430711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef911b-3424-4dc6-9ed9-42ea30724cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:49.431358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef911b-3424-4dc6-9ed9-42ea30724cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.431640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef911b-3424-4dc6-9ed9-42ea30724cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.431913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef911b-3424-4dc6-9ed9-42ea30724cfa admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.432704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef911b-3424-4dc6-9ed9-42ea30724cfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:12:49.433037 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef911b-3424-4dc6-9ed9-42ea30724cfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:49.434631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef911b-3424-4dc6-9ed9-42ea30724cfa admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:49.445994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1309/2623] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:12:49.450400 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352862f-3a7e-47ac-bcea-528d88604bea None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:49.459610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352862f-3a7e-47ac-bcea-528d88604bea None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:49.464900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f94078-8a34-4e0c-860b-9badefd8f0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:49.465558 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f94078-8a34-4e0c-860b-9badefd8f0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.465847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f94078-8a34-4e0c-860b-9badefd8f0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.466127 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f94078-8a34-4e0c-860b-9badefd8f0b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.466818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f94078-8a34-4e0c-860b-9badefd8f0b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:49.468745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f94078-8a34-4e0c-860b-9badefd8f0b1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:49.496447 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1315/2624] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:12:49 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:49.499344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef911b-3424-4dc6-9ed9-42ea30724cfa admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:49.509243 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef911b-3424-4dc6-9ed9-42ea30724cfa admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:49.512770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:49.568852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae99618-79e6-4a0a-8db5-c89f7f75a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:49.569531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae99618-79e6-4a0a-8db5-c89f7f75a84f admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.569813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae99618-79e6-4a0a-8db5-c89f7f75a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.570086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae99618-79e6-4a0a-8db5-c89f7f75a84f admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.570778 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae99618-79e6-4a0a-8db5-c89f7f75a84f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:49.572715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae99618-79e6-4a0a-8db5-c89f7f75a84f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:49.593834 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1310/2625] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:12:49 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:12:49.597991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f94078-8a34-4e0c-860b-9badefd8f0b1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:49.608732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f94078-8a34-4e0c-860b-9badefd8f0b1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:49.613830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5c543-e641-4a11-be5c-f239b356951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:49.614525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5c543-e641-4a11-be5c-f239b356951a admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.614811 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5c543-e641-4a11-be5c-f239b356951a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.615086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5c543-e641-4a11-be5c-f239b356951a admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.615838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5c543-e641-4a11-be5c-f239b356951a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:49.617481 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5c543-e641-4a11-be5c-f239b356951a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:49.627960 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1316/2626] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:12:49.632190 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae99618-79e6-4a0a-8db5-c89f7f75a84f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:49.642039 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae99618-79e6-4a0a-8db5-c89f7f75a84f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:49.646656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c3ec5-0723-4d6e-8671-1c1eee0f64a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:49.647314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c3ec5-0723-4d6e-8671-1c1eee0f64a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.647599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c3ec5-0723-4d6e-8671-1c1eee0f64a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.647877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c3ec5-0723-4d6e-8671-1c1eee0f64a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.648639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c3ec5-0723-4d6e-8671-1c1eee0f64a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:49.650227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c3ec5-0723-4d6e-8671-1c1eee0f64a5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:49.660965 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1311/2627] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:12:49.664896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5c543-e641-4a11-be5c-f239b356951a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:49.675218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5c543-e641-4a11-be5c-f239b356951a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:49.679824 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5b002-b9ac-4714-92f0-aa12a229de68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:49.680730 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b002-b9ac-4714-92f0-aa12a229de68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.681016 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b002-b9ac-4714-92f0-aa12a229de68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.681293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5b002-b9ac-4714-92f0-aa12a229de68 admin admin] PATH_INFO: `/v3/projects/48b4c040647e4c87bdc5c3710a482ba4/users/73d745660e5e45f58f402d7b8115786d/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.695446 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5b002-b9ac-4714-92f0-aa12a229de68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b4c040647e4c87bdc5c3710a482ba4, user_id=73d745660e5e45f58f402d7b8115786d, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:49.698129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5b002-b9ac-4714-92f0-aa12a229de68 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:49.711014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1317/2628] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:12:49 2022] PUT /identity/v3/projects/48b4c040647e4c87bdc5c3710a482ba4/users/73d745660e5e45f58f402d7b8115786d/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:12:49.722198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b91ef2-2e38-49bc-83a4-b06604968fd8 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.722435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b91ef2-2e38-49bc-83a4-b06604968fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.722720 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b91ef2-2e38-49bc-83a4-b06604968fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.749671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-b4b91ef2-2e38-49bc-83a4-b06604968fd8 None None] MFA Rules not processed for user `73d745660e5e45f58f402d7b8115786d`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:12:49.750866 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b4b91ef2-2e38-49bc-83a4-b06604968fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:49.815565 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1312/2629] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:12:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:12:49.826922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5b002-b9ac-4714-92f0-aa12a229de68 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:49.828635 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-52c5b002-b9ac-4714-92f0-aa12a229de68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:49.887924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5b002-b9ac-4714-92f0-aa12a229de68 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:49.892442 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415d1b8-e620-4376-b5bf-88414b42b609 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d1e01f6f82a4ebbb15d0cb55b963af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:49.893095 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415d1b8-e620-4376-b5bf-88414b42b609 service glance] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:49.893379 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415d1b8-e620-4376-b5bf-88414b42b609 service glance] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:49.893655 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415d1b8-e620-4376-b5bf-88414b42b609 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:49.902465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415d1b8-e620-4376-b5bf-88414b42b609 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:49.903926 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415d1b8-e620-4376-b5bf-88414b42b609 service glance] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:49.916053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1318/2630] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:12:52.552257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b91ef2-2e38-49bc-83a4-b06604968fd8 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:52.553824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b4b91ef2-2e38-49bc-83a4-b06604968fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:52.613648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b91ef2-2e38-49bc-83a4-b06604968fd8 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:52.617669 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3648f-a855-41cc-a99b-63b35952a4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:52.618452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3648f-a855-41cc-a99b-63b35952a4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:52.618813 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3648f-a855-41cc-a99b-63b35952a4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:52.619172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3648f-a855-41cc-a99b-63b35952a4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:52.621482 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fcd3648f-a855-41cc-a99b-63b35952a4f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:52.651530 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:52.683191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3648f-a855-41cc-a99b-63b35952a4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:52.685001 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3648f-a855-41cc-a99b-63b35952a4f0 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:52.696622 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1313/2631] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:53.789269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415d1b8-e620-4376-b5bf-88414b42b609 service glance] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:53.791371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4415d1b8-e620-4376-b5bf-88414b42b609 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:53.849357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:12:53.887164 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415d1b8-e620-4376-b5bf-88414b42b609 service glance] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:53.895826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1696a-cf7b-4678-b6e1-50d7e0e20679 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:53.896584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1696a-cf7b-4678-b6e1-50d7e0e20679 service nova] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:53.896889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1696a-cf7b-4678-b6e1-50d7e0e20679 service nova] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:53.897170 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1696a-cf7b-4678-b6e1-50d7e0e20679 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:53.899184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-13f1696a-cf7b-4678-b6e1-50d7e0e20679 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:53.913501 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1696a-cf7b-4678-b6e1-50d7e0e20679 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:53.915182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1696a-cf7b-4678-b6e1-50d7e0e20679 service nova] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:53.934637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1319/2632] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:12:58.029040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3648f-a855-41cc-a99b-63b35952a4f0 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:12:58.042584 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3648f-a855-41cc-a99b-63b35952a4f0 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:12:58.048505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb44ba5-278a-42c6-adc5-bf8ad3e548cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:12:58.049342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb44ba5-278a-42c6-adc5-bf8ad3e548cf service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:12:58.049683 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb44ba5-278a-42c6-adc5-bf8ad3e548cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:12:58.049969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb44ba5-278a-42c6-adc5-bf8ad3e548cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:12:58.052571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-cbb44ba5-278a-42c6-adc5-bf8ad3e548cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:12:58.154843 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb44ba5-278a-42c6-adc5-bf8ad3e548cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:12:58.156600 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb44ba5-278a-42c6-adc5-bf8ad3e548cf service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:12:58.170345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1314/2633] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:01.552033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1696a-cf7b-4678-b6e1-50d7e0e20679 service nova] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:01.553570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-13f1696a-cf7b-4678-b6e1-50d7e0e20679 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:01.657722 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1696a-cf7b-4678-b6e1-50d7e0e20679 service nova] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:01.669058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eacaf8-1223-441b-925d-d63bd11e3037 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:01.669878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eacaf8-1223-441b-925d-d63bd11e3037 service placement] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:01.670278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eacaf8-1223-441b-925d-d63bd11e3037 service placement] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:01.670581 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eacaf8-1223-441b-925d-d63bd11e3037 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:01.672634 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c4eacaf8-1223-441b-925d-d63bd11e3037 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:01.685767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eacaf8-1223-441b-925d-d63bd11e3037 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:01.687693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eacaf8-1223-441b-925d-d63bd11e3037 service placement] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:01.698503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1320/2634] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:02.547316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb44ba5-278a-42c6-adc5-bf8ad3e548cf service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:02.556115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb44ba5-278a-42c6-adc5-bf8ad3e548cf service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:02.559978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98912b-e02e-4b72-9140-549a1bb85c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:02.561084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98912b-e02e-4b72-9140-549a1bb85c10 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:02.561369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98912b-e02e-4b72-9140-549a1bb85c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:02.561655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98912b-e02e-4b72-9140-549a1bb85c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:02.563490 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ce98912b-e02e-4b72-9140-549a1bb85c10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:02.576309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98912b-e02e-4b72-9140-549a1bb85c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:02.577788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98912b-e02e-4b72-9140-549a1bb85c10 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:02.587589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1315/2635] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:05.020966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eacaf8-1223-441b-925d-d63bd11e3037 service placement] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:05.022535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-c4eacaf8-1223-441b-925d-d63bd11e3037 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:05.048821 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eacaf8-1223-441b-925d-d63bd11e3037 service placement] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:05.070546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:05.070546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:05.070546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:05.070546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] PATH_INFO: `/v3/users/00c126e7436243559b25981f5ca7c74e` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:05.071133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c126e7436243559b25981f5ca7c74e)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:05.071133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:05.131525 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c126e7436243559b25981f5ca7c74e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.131983 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c126e7436243559b25981f5ca7c74e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.141643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c126e7436243559b25981f5ca7c74e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.142338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c126e7436243559b25981f5ca7c74e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.158290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c126e7436243559b25981f5ca7c74e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.172609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1321/2636] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:13:05 2022] DELETE /identity/v3/users/00c126e7436243559b25981f5ca7c74e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:05.361468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98912b-e02e-4b72-9140-549a1bb85c10 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:05.373492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98912b-e02e-4b72-9140-549a1bb85c10 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:05.478346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:05.479311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:05.479670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:05.480016 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] PATH_INFO: `/v3/projects/43e5d438bb1f4eb28d3f3f68d77b5dee` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:05.481503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43e5d438bb1f4eb28d3f3f68d77b5dee)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:05.484247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:05.534733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43e5d438bb1f4eb28d3f3f68d77b5dee', 'request_id': 'req-28518090-292f-4476-839d-d57335cbba07'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.537637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:05.543960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43e5d438bb1f4eb28d3f3f68d77b5dee', 'request_id': 'req-28518090-292f-4476-839d-d57335cbba07'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.555768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43e5d438bb1f4eb28d3f3f68d77b5dee', 'request_id': 'req-28518090-292f-4476-839d-d57335cbba07'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.573141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] The token cache is being invalidate because project 43e5d438bb1f4eb28d3f3f68d77b5dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:13:05.573141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.574522 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.579421 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1316/2637] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:13:05 2022] DELETE /identity/v3/projects/43e5d438bb1f4eb28d3f3f68d77b5dee => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:05.580235 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:05.581714 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:05.596346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6228d5-6b6c-443c-9271-566e89b74e30 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:05.639467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:05.679057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:05.679976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:05.680266 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:05.680586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] PATH_INFO: `/v3/users/4828765f878648bdacb251e41bdf21a0` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:05.681791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4828765f878648bdacb251e41bdf21a0)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:05.683778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:05.733228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4828765f878648bdacb251e41bdf21a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.734660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4828765f878648bdacb251e41bdf21a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.744166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4828765f878648bdacb251e41bdf21a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.745092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4828765f878648bdacb251e41bdf21a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.775435 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4828765f878648bdacb251e41bdf21a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:05.806134 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1322/2638] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:13:05 2022] DELETE /identity/v3/users/4828765f878648bdacb251e41bdf21a0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:06.014259 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:06.015810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:06.033845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28518090-292f-4476-839d-d57335cbba07 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:06.136029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:06.137029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:06.137390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:06.137745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] PATH_INFO: `/v3/projects/ba73173cd15a45f7a29d6b5112a33770` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:06.139312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba73173cd15a45f7a29d6b5112a33770)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:06.141409 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:06.187808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:06.189317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:06.205936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b231ed7-3d2a-466e-9d4d-486db3a900c7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:06.213405 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:06.214449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:06.215072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:06.215516 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] PATH_INFO: `/v3/users/a3b4081bebe74c839f5aa214275210b4` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:06.217710 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b4081bebe74c839f5aa214275210b4)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:06.220780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:06.240927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba73173cd15a45f7a29d6b5112a33770', 'request_id': 'req-6717b993-c8a1-4749-9023-8e6c4b71214e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.250668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba73173cd15a45f7a29d6b5112a33770', 'request_id': 'req-6717b993-c8a1-4749-9023-8e6c4b71214e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.259215 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba73173cd15a45f7a29d6b5112a33770', 'request_id': 'req-6717b993-c8a1-4749-9023-8e6c4b71214e'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.273226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] The token cache is being invalidate because project ba73173cd15a45f7a29d6b5112a33770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:13:06.273731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.274808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.280014 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1317/2639] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:13:06 2022] DELETE /identity/v3/projects/ba73173cd15a45f7a29d6b5112a33770 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:13:06.301630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b4081bebe74c839f5aa214275210b4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.302616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b4081bebe74c839f5aa214275210b4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.316316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b4081bebe74c839f5aa214275210b4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.317286 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b4081bebe74c839f5aa214275210b4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.335244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-e48f9442-7536-4c6b-bec8-b39f6a988ea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b4081bebe74c839f5aa214275210b4'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.356528 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1323/2640] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:13:06 2022] DELETE /identity/v3/users/a3b4081bebe74c839f5aa214275210b4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:06.568064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:06.569646 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:06.649329 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717b993-c8a1-4749-9023-8e6c4b71214e admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:06.662438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:06.708106 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:06.709005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:06.709299 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:06.709576 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] PATH_INFO: `/v3/projects/7daf0f9fd80c464ca4fbf7f2e19fccbd` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:06.710759 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7daf0f9fd80c464ca4fbf7f2e19fccbd)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:06.712705 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:06.758740 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7daf0f9fd80c464ca4fbf7f2e19fccbd', 'request_id': 'req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.765297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7daf0f9fd80c464ca4fbf7f2e19fccbd', 'request_id': 'req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.773290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7daf0f9fd80c464ca4fbf7f2e19fccbd', 'request_id': 'req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.786089 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] The token cache is being invalidate because project 7daf0f9fd80c464ca4fbf7f2e19fccbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:13:06.786380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.787056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-c08b5cdb-44e4-4d97-a6c8-6c6983ed29fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:06.788587 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1318/2641] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:13:06 2022] DELETE /identity/v3/projects/7daf0f9fd80c464ca4fbf7f2e19fccbd => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:06.793608 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb637-15aa-4c53-8c67-c80ae487e2ab None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:06.793900 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb637-15aa-4c53-8c67-c80ae487e2ab None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:06.794761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb637-15aa-4c53-8c67-c80ae487e2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:06.817705 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-a7dfb637-15aa-4c53-8c67-c80ae487e2ab None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:06.818942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a7dfb637-15aa-4c53-8c67-c80ae487e2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:06.840779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:06.889543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1324/2642] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:13:06.897117 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9ac1a-37f2-4b09-bf58-64134a85f0fe None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:06.897407 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9ac1a-37f2-4b09-bf58-64134a85f0fe None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:06.897686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9ac1a-37f2-4b09-bf58-64134a85f0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:06.918913 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-4ac9ac1a-37f2-4b09-bf58-64134a85f0fe None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:06.920334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-4ac9ac1a-37f2-4b09-bf58-64134a85f0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:06.928314 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1319/2643] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:06.932601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfb637-15aa-4c53-8c67-c80ae487e2ab None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:06.943211 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfb637-15aa-4c53-8c67-c80ae487e2ab None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:06.947788 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eda873-7b7a-4dc5-a94a-30f18c69b395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:06.948466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eda873-7b7a-4dc5-a94a-30f18c69b395 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:06.948754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eda873-7b7a-4dc5-a94a-30f18c69b395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:06.949034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eda873-7b7a-4dc5-a94a-30f18c69b395 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:06.949762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eda873-7b7a-4dc5-a94a-30f18c69b395 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:13:06.950072 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eda873-7b7a-4dc5-a94a-30f18c69b395 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:06.952371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eda873-7b7a-4dc5-a94a-30f18c69b395 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:06.963782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1325/2644] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 08 02:13:06.968027 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9ac1a-37f2-4b09-bf58-64134a85f0fe None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:06.977355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9ac1a-37f2-4b09-bf58-64134a85f0fe None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:06.982620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b347d-c4cc-4d02-b582-113dfd235298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:06.983275 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b347d-c4cc-4d02-b582-113dfd235298 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:06.983566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b347d-c4cc-4d02-b582-113dfd235298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:06.983844 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b347d-c4cc-4d02-b582-113dfd235298 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:06.984573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b347d-c4cc-4d02-b582-113dfd235298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:06.986472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b347d-c4cc-4d02-b582-113dfd235298 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:07.011520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1320/2645] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:13:06 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:13:07.016278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eda873-7b7a-4dc5-a94a-30f18c69b395 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:07.026296 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eda873-7b7a-4dc5-a94a-30f18c69b395 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:07.083466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f08b12-fb38-457d-8b26-f66f12236ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:07.084064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f08b12-fb38-457d-8b26-f66f12236ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:07.084359 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f08b12-fb38-457d-8b26-f66f12236ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:07.084682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f08b12-fb38-457d-8b26-f66f12236ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:07.085384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f08b12-fb38-457d-8b26-f66f12236ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:07.087307 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f08b12-fb38-457d-8b26-f66f12236ca5 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:07.107225 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1326/2646] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:13:07 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:13:07.111297 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b347d-c4cc-4d02-b582-113dfd235298 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:07.121767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b347d-c4cc-4d02-b582-113dfd235298 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:07.126556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d65b9-e8de-42c0-94b1-3288985cfa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:07.127220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d65b9-e8de-42c0-94b1-3288985cfa04 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:07.127501 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d65b9-e8de-42c0-94b1-3288985cfa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:07.127774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d65b9-e8de-42c0-94b1-3288985cfa04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:07.128533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d65b9-e8de-42c0-94b1-3288985cfa04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:07.130124 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d65b9-e8de-42c0-94b1-3288985cfa04 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:07.140992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1321/2647] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:13:07.144861 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f08b12-fb38-457d-8b26-f66f12236ca5 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:07.155786 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f08b12-fb38-457d-8b26-f66f12236ca5 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:07.160686 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320889f-c0b6-4406-82d6-47f599c0d63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:07.161340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320889f-c0b6-4406-82d6-47f599c0d63b admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:07.161626 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320889f-c0b6-4406-82d6-47f599c0d63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:07.161904 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320889f-c0b6-4406-82d6-47f599c0d63b admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:07.162633 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320889f-c0b6-4406-82d6-47f599c0d63b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:07.164228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320889f-c0b6-4406-82d6-47f599c0d63b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:07.174303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1327/2648] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:13:07.178290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d65b9-e8de-42c0-94b1-3288985cfa04 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:07.188723 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d65b9-e8de-42c0-94b1-3288985cfa04 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:07.193585 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84b162-1b81-4698-ad2a-235afb531d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:07.194461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84b162-1b81-4698-ad2a-235afb531d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:07.194747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84b162-1b81-4698-ad2a-235afb531d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:07.195024 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84b162-1b81-4698-ad2a-235afb531d58 admin admin] PATH_INFO: `/v3/projects/879e3eacb3bc4b69891677c86bbd5de3/users/92de4ab63f6447eabefe34f65c71d918/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:07.210134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84b162-1b81-4698-ad2a-235afb531d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879e3eacb3bc4b69891677c86bbd5de3, user_id=92de4ab63f6447eabefe34f65c71d918, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:07.212830 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84b162-1b81-4698-ad2a-235afb531d58 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:07.226181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1322/2649] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:13:07 2022] PUT /identity/v3/projects/879e3eacb3bc4b69891677c86bbd5de3/users/92de4ab63f6447eabefe34f65c71d918/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:07.231144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbed6f2-d240-4831-a0c1-64ff77d6efee None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:07.231641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbed6f2-d240-4831-a0c1-64ff77d6efee None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:07.231763 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbed6f2-d240-4831-a0c1-64ff77d6efee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:07.249193 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7dbed6f2-d240-4831-a0c1-64ff77d6efee None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:07.250391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7dbed6f2-d240-4831-a0c1-64ff77d6efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:07.309630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1328/2650] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:07.319400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84b162-1b81-4698-ad2a-235afb531d58 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:07.321120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1b84b162-1b81-4698-ad2a-235afb531d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:07.367776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:07.380651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84b162-1b81-4698-ad2a-235afb531d58 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:07.384665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c2d14-a5ad-49fd-a8df-96d0a026a389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:07.385311 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2d14-a5ad-49fd-a8df-96d0a026a389 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:07.385592 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2d14-a5ad-49fd-a8df-96d0a026a389 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:07.385865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2d14-a5ad-49fd-a8df-96d0a026a389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:07.395487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c2d14-a5ad-49fd-a8df-96d0a026a389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:07.396952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c2d14-a5ad-49fd-a8df-96d0a026a389 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:07.406478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1323/2651] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:07.664010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbed6f2-d240-4831-a0c1-64ff77d6efee None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:07.665452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7dbed6f2-d240-4831-a0c1-64ff77d6efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:07.677787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbed6f2-d240-4831-a0c1-64ff77d6efee None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:07.683529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:07.684413 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:07.685233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:07.685624 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] PATH_INFO: `/v3/users/73d745660e5e45f58f402d7b8115786d` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:07.687115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d745660e5e45f58f402d7b8115786d)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:07.689112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:07.722328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:07.755630 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d745660e5e45f58f402d7b8115786d'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:07.756519 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d745660e5e45f58f402d7b8115786d'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:07.764893 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d745660e5e45f58f402d7b8115786d'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:07.766123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d745660e5e45f58f402d7b8115786d'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:07.783236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d745660e5e45f58f402d7b8115786d'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:07.798882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1329/2652] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:13:07 2022] DELETE /identity/v3/users/73d745660e5e45f58f402d7b8115786d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:13:07.803437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1553a-b863-46c2-b7d7-8ceecf5635a1 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:07.803932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1553a-b863-46c2-b7d7-8ceecf5635a1 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:07.804128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1553a-b863-46c2-b7d7-8ceecf5635a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:07.822471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-e8d1553a-b863-46c2-b7d7-8ceecf5635a1 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:07.823783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e8d1553a-b863-46c2-b7d7-8ceecf5635a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:07.887263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1324/2653] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:07.897505 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:07.898941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:07.962123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527c9c7-941d-4023-bf07-932f629b9a38 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:07.966852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27390a-2404-4179-a40c-cf4f0a18b301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:07.967680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27390a-2404-4179-a40c-cf4f0a18b301 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:07.967961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27390a-2404-4179-a40c-cf4f0a18b301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:07.968244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27390a-2404-4179-a40c-cf4f0a18b301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:07.978674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27390a-2404-4179-a40c-cf4f0a18b301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:07.980133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27390a-2404-4179-a40c-cf4f0a18b301 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:07.990609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1330/2654] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:08.338417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1553a-b863-46c2-b7d7-8ceecf5635a1 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:08.339927 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e8d1553a-b863-46c2-b7d7-8ceecf5635a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:08.359175 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1553a-b863-46c2-b7d7-8ceecf5635a1 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:08.365851 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:08.366731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:08.367019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:08.367299 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] PATH_INFO: `/v3/projects/48b4c040647e4c87bdc5c3710a482ba4` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:08.368583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48b4c040647e4c87bdc5c3710a482ba4)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:08.371470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:08.448381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b4c040647e4c87bdc5c3710a482ba4', 'request_id': 'req-cef6e23c-988b-429d-ac21-a0a8552c2a74'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:08.456886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b4c040647e4c87bdc5c3710a482ba4', 'request_id': 'req-cef6e23c-988b-429d-ac21-a0a8552c2a74'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:08.467182 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b4c040647e4c87bdc5c3710a482ba4', 'request_id': 'req-cef6e23c-988b-429d-ac21-a0a8552c2a74'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:08.478631 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] The token cache is being invalidate because project 48b4c040647e4c87bdc5c3710a482ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:13:08.478918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:08.480179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cef6e23c-988b-429d-ac21-a0a8552c2a74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:08.482493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1325/2655] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:13:08 2022] DELETE /identity/v3/projects/48b4c040647e4c87bdc5c3710a482ba4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:08.486991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164a8d4-246c-40bd-8949-5d9af8b2fb7e None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:08.487497 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164a8d4-246c-40bd-8949-5d9af8b2fb7e None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:08.487583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164a8d4-246c-40bd-8949-5d9af8b2fb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:08.507669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-8164a8d4-246c-40bd-8949-5d9af8b2fb7e None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:08.509133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8164a8d4-246c-40bd-8949-5d9af8b2fb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:08.588436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1331/2656] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:08.590833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0142d5-bc29-4070-ab48-022d5200e945 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:08.591400 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0142d5-bc29-4070-ab48-022d5200e945 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:08.591832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0142d5-bc29-4070-ab48-022d5200e945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:08.612165 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ed0142d5-bc29-4070-ab48-022d5200e945 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:08.613457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ed0142d5-bc29-4070-ab48-022d5200e945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:08.622456 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1326/2657] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:13:08.627424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164a8d4-246c-40bd-8949-5d9af8b2fb7e None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:08.643638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164a8d4-246c-40bd-8949-5d9af8b2fb7e None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:08.651924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8dc1b-5305-401b-8c0c-2274ebe152ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:08.652800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8dc1b-5305-401b-8c0c-2274ebe152ca admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:08.653089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8dc1b-5305-401b-8c0c-2274ebe152ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:08.653554 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8dc1b-5305-401b-8c0c-2274ebe152ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:08.654573 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8dc1b-5305-401b-8c0c-2274ebe152ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:13:08.654884 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8dc1b-5305-401b-8c0c-2274ebe152ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:08.656627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8dc1b-5305-401b-8c0c-2274ebe152ca admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:08.670586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1332/2658] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:13:08.674736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0142d5-bc29-4070-ab48-022d5200e945 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:08.684821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0142d5-bc29-4070-ab48-022d5200e945 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:08.692442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07bab5-ad6c-413c-92f0-1d9478752d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:08.693203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07bab5-ad6c-413c-92f0-1d9478752d28 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:08.693574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07bab5-ad6c-413c-92f0-1d9478752d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:08.694263 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07bab5-ad6c-413c-92f0-1d9478752d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:08.694771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07bab5-ad6c-413c-92f0-1d9478752d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:08.697032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07bab5-ad6c-413c-92f0-1d9478752d28 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:08.741281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1327/2659] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:13:08 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:13:08.745787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8dc1b-5305-401b-8c0c-2274ebe152ca admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:08.757289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8dc1b-5305-401b-8c0c-2274ebe152ca admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:08.802987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:08.847512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8fa2d-42fb-4a68-a117-5aa8240cbc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:08.848212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8fa2d-42fb-4a68-a117-5aa8240cbc87 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:08.848540 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8fa2d-42fb-4a68-a117-5aa8240cbc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:08.848826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8fa2d-42fb-4a68-a117-5aa8240cbc87 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:08.849729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8fa2d-42fb-4a68-a117-5aa8240cbc87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:08.851726 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8fa2d-42fb-4a68-a117-5aa8240cbc87 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:08.895552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1333/2660] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:13:08 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:13:08.896046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07bab5-ad6c-413c-92f0-1d9478752d28 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:08.930803 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07bab5-ad6c-413c-92f0-1d9478752d28 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:08.936373 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72465d40-a339-4110-8ffc-d441c102f943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:08.937075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72465d40-a339-4110-8ffc-d441c102f943 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:08.937361 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72465d40-a339-4110-8ffc-d441c102f943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:08.937640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72465d40-a339-4110-8ffc-d441c102f943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:08.938404 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72465d40-a339-4110-8ffc-d441c102f943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:08.940019 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72465d40-a339-4110-8ffc-d441c102f943 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:08.951527 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1328/2661] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:13:08.955486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8fa2d-42fb-4a68-a117-5aa8240cbc87 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:08.973510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8fa2d-42fb-4a68-a117-5aa8240cbc87 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:08.979935 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9e551-0ff2-42c6-95a6-b4f023e87e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:08.980503 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9e551-0ff2-42c6-95a6-b4f023e87e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:08.980795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9e551-0ff2-42c6-95a6-b4f023e87e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:08.981079 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9e551-0ff2-42c6-95a6-b4f023e87e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:08.981928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9e551-0ff2-42c6-95a6-b4f023e87e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:08.983559 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9e551-0ff2-42c6-95a6-b4f023e87e71 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:08.994425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1334/2662] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:13:08.999272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72465d40-a339-4110-8ffc-d441c102f943 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:09.015898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72465d40-a339-4110-8ffc-d441c102f943 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:09.021096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cc6de-533a-4147-9563-7708ca969104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:09.022107 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cc6de-533a-4147-9563-7708ca969104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:09.022458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cc6de-533a-4147-9563-7708ca969104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:09.022806 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cc6de-533a-4147-9563-7708ca969104 admin admin] PATH_INFO: `/v3/projects/ed826435ba514acc99aa5b284fed3a48/users/645a17a1e93047369429c764427c2ff1/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:09.046814 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cc6de-533a-4147-9563-7708ca969104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed826435ba514acc99aa5b284fed3a48, user_id=645a17a1e93047369429c764427c2ff1, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:09.049670 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cc6de-533a-4147-9563-7708ca969104 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:09.066888 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1329/2663] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:13:08 2022] PUT /identity/v3/projects/ed826435ba514acc99aa5b284fed3a48/users/645a17a1e93047369429c764427c2ff1/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:09.078116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113a2e5-d703-4db1-85df-822faba1b844 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:09.078981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113a2e5-d703-4db1-85df-822faba1b844 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:09.079119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113a2e5-d703-4db1-85df-822faba1b844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:09.112745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-6113a2e5-d703-4db1-85df-822faba1b844 None None] MFA Rules not processed for user `645a17a1e93047369429c764427c2ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:09.114149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6113a2e5-d703-4db1-85df-822faba1b844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:09.186581 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1335/2664] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:09.197417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cc6de-533a-4147-9563-7708ca969104 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:09.199110 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e24cc6de-533a-4147-9563-7708ca969104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:09.209219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:09.267924 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cc6de-533a-4147-9563-7708ca969104 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:09.272171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2133ee-b588-4c83-ae6f-0e6f44ceb72e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:09.272336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2133ee-b588-4c83-ae6f-0e6f44ceb72e service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:09.272561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2133ee-b588-4c83-ae6f-0e6f44ceb72e service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:09.272841 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2133ee-b588-4c83-ae6f-0e6f44ceb72e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:09.282385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2133ee-b588-4c83-ae6f-0e6f44ceb72e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:09.283835 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2133ee-b588-4c83-ae6f-0e6f44ceb72e service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:09.293689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1330/2665] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:12.307771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03d5d6-10ff-475f-bbb4-38d5687e2cc6 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:12.308250 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03d5d6-10ff-475f-bbb4-38d5687e2cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:12.308376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03d5d6-10ff-475f-bbb4-38d5687e2cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:12.335739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-ab03d5d6-10ff-475f-bbb4-38d5687e2cc6 None None] MFA Rules not processed for user `92de4ab63f6447eabefe34f65c71d918`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:12.336979 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ab03d5d6-10ff-475f-bbb4-38d5687e2cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:12.405333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1336/2666] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:12.420732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2133ee-b588-4c83-ae6f-0e6f44ceb72e service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:12.422553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9c2133ee-b588-4c83-ae6f-0e6f44ceb72e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:12.503630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2133ee-b588-4c83-ae6f-0e6f44ceb72e service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:12.509191 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdb706-b7e4-48bd-b083-d601a8f076a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:12.509844 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdb706-b7e4-48bd-b083-d601a8f076a7 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:12.510130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdb706-b7e4-48bd-b083-d601a8f076a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:12.510410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdb706-b7e4-48bd-b083-d601a8f076a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:12.518854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdb706-b7e4-48bd-b083-d601a8f076a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:12.520479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdb706-b7e4-48bd-b083-d601a8f076a7 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:12.529333 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1331/2667] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:12.714561 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03d5d6-10ff-475f-bbb4-38d5687e2cc6 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:12.780433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:12.860700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03d5d6-10ff-475f-bbb4-38d5687e2cc6 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:12.866489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:12.867778 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:12.868241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:12.868761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] PATH_INFO: `/v3/users/645a17a1e93047369429c764427c2ff1` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:12.871013 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] RBAC: Authorizing `identity:delete_user(user_id=645a17a1e93047369429c764427c2ff1)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:12.874055 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:12.960303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '645a17a1e93047369429c764427c2ff1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:12.962256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '645a17a1e93047369429c764427c2ff1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:12.986754 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '645a17a1e93047369429c764427c2ff1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:12.987385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '645a17a1e93047369429c764427c2ff1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:13.005288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '645a17a1e93047369429c764427c2ff1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:13.023781 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1337/2668] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:13:12 2022] DELETE /identity/v3/users/645a17a1e93047369429c764427c2ff1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:13.029395 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0427992-e292-4384-9b82-1f0cdcd26859 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:13.029686 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0427992-e292-4384-9b82-1f0cdcd26859 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:13.029966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0427992-e292-4384-9b82-1f0cdcd26859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:13.053616 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-d0427992-e292-4384-9b82-1f0cdcd26859 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:13.054989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d0427992-e292-4384-9b82-1f0cdcd26859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:13.136206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1332/2669] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:13.147571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:13.149155 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:13.218051 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493b063-7442-4e8f-99a5-92260820a964 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:13.223383 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e564f-b13d-42ca-8631-bddedf51d6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:13.224351 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e564f-b13d-42ca-8631-bddedf51d6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:13.225515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e564f-b13d-42ca-8631-bddedf51d6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:13.226566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e564f-b13d-42ca-8631-bddedf51d6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:13.241306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e564f-b13d-42ca-8631-bddedf51d6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:13.243322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e564f-b13d-42ca-8631-bddedf51d6d1 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:13.260408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1338/2670] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:13:13.588260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0427992-e292-4384-9b82-1f0cdcd26859 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:13.592940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d0427992-e292-4384-9b82-1f0cdcd26859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:13.609533 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0427992-e292-4384-9b82-1f0cdcd26859 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:13.617063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:13.618127 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:13.618598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:13.618873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] PATH_INFO: `/v3/projects/ed826435ba514acc99aa5b284fed3a48` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:13.620246 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed826435ba514acc99aa5b284fed3a48)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:13.622404 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:13.671548 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:13.689624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed826435ba514acc99aa5b284fed3a48', 'request_id': 'req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:13.698467 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed826435ba514acc99aa5b284fed3a48', 'request_id': 'req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:13.708017 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed826435ba514acc99aa5b284fed3a48', 'request_id': 'req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:13.724133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] The token cache is being invalidate because project ed826435ba514acc99aa5b284fed3a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:13:13.725023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:13.725676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ac92c5ee-eb5e-42e6-baf4-b9b01ea4d4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:13.727829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1333/2671] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:13:13 2022] DELETE /identity/v3/projects/ed826435ba514acc99aa5b284fed3a48 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:13.733089 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b973a-c37c-414e-b39a-bef9fb436156 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:13.733607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b973a-c37c-414e-b39a-bef9fb436156 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:13.733976 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b973a-c37c-414e-b39a-bef9fb436156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:13.753320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b13b973a-c37c-414e-b39a-bef9fb436156 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:13.754676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b13b973a-c37c-414e-b39a-bef9fb436156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:13.817649 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:13.843241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1339/2672] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:13.852029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a694b-5516-4dc4-a8d0-db586238385d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:13.852316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a694b-5516-4dc4-a8d0-db586238385d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:13.852654 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a694b-5516-4dc4-a8d0-db586238385d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:13.873214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-af4a694b-5516-4dc4-a8d0-db586238385d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:13.874507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-af4a694b-5516-4dc4-a8d0-db586238385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:13.893838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b973a-c37c-414e-b39a-bef9fb436156 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:13.901288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1334/2673] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:13.905734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b973a-c37c-414e-b39a-bef9fb436156 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:13.911218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307954f-684f-410b-87f2-dcbe36d0df3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:13.912328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307954f-684f-410b-87f2-dcbe36d0df3a admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:13.912656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307954f-684f-410b-87f2-dcbe36d0df3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:13.912955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307954f-684f-410b-87f2-dcbe36d0df3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:13.913699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307954f-684f-410b-87f2-dcbe36d0df3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:13:13.914008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307954f-684f-410b-87f2-dcbe36d0df3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:13.915661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307954f-684f-410b-87f2-dcbe36d0df3a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:13.930946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1340/2674] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:13:13.935832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a694b-5516-4dc4-a8d0-db586238385d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:13.946075 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a694b-5516-4dc4-a8d0-db586238385d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:13.952788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25db9e9-5dfb-4ef3-8423-031247b74820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:13.953833 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25db9e9-5dfb-4ef3-8423-031247b74820 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:13.954336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25db9e9-5dfb-4ef3-8423-031247b74820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:13.954797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25db9e9-5dfb-4ef3-8423-031247b74820 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:13.955932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25db9e9-5dfb-4ef3-8423-031247b74820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:13.958345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25db9e9-5dfb-4ef3-8423-031247b74820 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:14.006867 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1335/2675] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:13:13 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:13:14.009418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307954f-684f-410b-87f2-dcbe36d0df3a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:14.021800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307954f-684f-410b-87f2-dcbe36d0df3a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:14.119321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108abbee-9d77-4b07-9116-9d26e545c371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:14.120133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108abbee-9d77-4b07-9116-9d26e545c371 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:14.120567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108abbee-9d77-4b07-9116-9d26e545c371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:14.121183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108abbee-9d77-4b07-9116-9d26e545c371 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:14.122615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108abbee-9d77-4b07-9116-9d26e545c371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:14.125367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108abbee-9d77-4b07-9116-9d26e545c371 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:14.160809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f7bc2-c31a-4da0-80bf-eb8223957efe None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:14.161094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f7bc2-c31a-4da0-80bf-eb8223957efe None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:14.161371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f7bc2-c31a-4da0-80bf-eb8223957efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:14.162576 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1341/2676] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:13:14 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:13:14.167179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108abbee-9d77-4b07-9116-9d26e545c371 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:14.187115 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108abbee-9d77-4b07-9116-9d26e545c371 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:14.191829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5d9f7bc2-c31a-4da0-80bf-eb8223957efe None None] MFA Rules not processed for user `92de4ab63f6447eabefe34f65c71d918`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:14.195130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff2163-ccda-4682-bfbd-48071888600f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:14.196035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff2163-ccda-4682-bfbd-48071888600f admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:14.197642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff2163-ccda-4682-bfbd-48071888600f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:14.198113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff2163-ccda-4682-bfbd-48071888600f admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:14.199086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff2163-ccda-4682-bfbd-48071888600f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:14.200096 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5d9f7bc2-c31a-4da0-80bf-eb8223957efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:14.203756 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff2163-ccda-4682-bfbd-48071888600f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:14.223813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1342/2677] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:13:14.224826 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff2163-ccda-4682-bfbd-48071888600f admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:14.240057 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff2163-ccda-4682-bfbd-48071888600f admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:14.246031 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2f62a-72b2-48a1-ac57-c87a7ae72b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:14.246830 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2f62a-72b2-48a1-ac57-c87a7ae72b85 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:14.247657 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2f62a-72b2-48a1-ac57-c87a7ae72b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:14.247947 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2f62a-72b2-48a1-ac57-c87a7ae72b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:14.257038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2f62a-72b2-48a1-ac57-c87a7ae72b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:14.267158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2f62a-72b2-48a1-ac57-c87a7ae72b85 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:14.287318 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1343/2678] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:13:14.291254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2f62a-72b2-48a1-ac57-c87a7ae72b85 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:14.310522 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2f62a-72b2-48a1-ac57-c87a7ae72b85 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:14.313524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1336/2679] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:14.336746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b92477-eb79-4a1a-ba90-299d3f0dceb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:14.337567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b92477-eb79-4a1a-ba90-299d3f0dceb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:14.337851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b92477-eb79-4a1a-ba90-299d3f0dceb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:14.338125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b92477-eb79-4a1a-ba90-299d3f0dceb7 admin admin] PATH_INFO: `/v3/projects/64c0ca427b564e258fb670f0b9ce6cb4/users/6a5d87b148964841977ddba4f9b9e7ca/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:14.356166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b92477-eb79-4a1a-ba90-299d3f0dceb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c0ca427b564e258fb670f0b9ce6cb4, user_id=6a5d87b148964841977ddba4f9b9e7ca, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:14.366299 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b92477-eb79-4a1a-ba90-299d3f0dceb7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:14.400484 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1344/2680] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:13:14 2022] PUT /identity/v3/projects/64c0ca427b564e258fb670f0b9ce6cb4/users/6a5d87b148964841977ddba4f9b9e7ca/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:14.405854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5033fb-556b-465c-8803-6b105f69a903 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:14.406349 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5033fb-556b-465c-8803-6b105f69a903 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:14.406750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5033fb-556b-465c-8803-6b105f69a903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:14.440641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-fc5033fb-556b-465c-8803-6b105f69a903 None None] MFA Rules not processed for user `6a5d87b148964841977ddba4f9b9e7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:14.442339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fc5033fb-556b-465c-8803-6b105f69a903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:14.528076 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1337/2681] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:13:14.543874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b92477-eb79-4a1a-ba90-299d3f0dceb7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:14.547716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-20b92477-eb79-4a1a-ba90-299d3f0dceb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:14.627762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b92477-eb79-4a1a-ba90-299d3f0dceb7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:14.633384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11ebcb-3438-4f5d-81ba-d36a45fa80e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d1e01f6f82a4ebbb15d0cb55b963af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:14.633384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ebcb-3438-4f5d-81ba-d36a45fa80e8 service glance] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:14.633384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ebcb-3438-4f5d-81ba-d36a45fa80e8 service glance] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:14.633384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ebcb-3438-4f5d-81ba-d36a45fa80e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:14.643747 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11ebcb-3438-4f5d-81ba-d36a45fa80e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:14.645339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11ebcb-3438-4f5d-81ba-d36a45fa80e8 service glance] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:14.656990 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1345/2682] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:24.523994 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=82,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:24.652277 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=85,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:29.592301 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5033fb-556b-465c-8803-6b105f69a903 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:29.593735 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fc5033fb-556b-465c-8803-6b105f69a903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:29.666427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5033fb-556b-465c-8803-6b105f69a903 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:29.671892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fc0b6-ea81-4a0e-90a4-9ac43390e4c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:29.672582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fc0b6-ea81-4a0e-90a4-9ac43390e4c8 service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:29.672914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fc0b6-ea81-4a0e-90a4-9ac43390e4c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:29.673193 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fc0b6-ea81-4a0e-90a4-9ac43390e4c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:29.675058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9c7fc0b6-ea81-4a0e-90a4-9ac43390e4c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:29.687014 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fc0b6-ea81-4a0e-90a4-9ac43390e4c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:29.688665 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fc0b6-ea81-4a0e-90a4-9ac43390e4c8 service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:29.698907 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1338/2683] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:39.694579 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=32 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:40.420007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11ebcb-3438-4f5d-81ba-d36a45fa80e8 service glance] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:40.483184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11ebcb-3438-4f5d-81ba-d36a45fa80e8 service glance] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:40.487737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:40.488627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:40.488917 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:40.489196 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] PATH_INFO: `/v3/users/6a5d87b148964841977ddba4f9b9e7ca` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:40.490384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a5d87b148964841977ddba4f9b9e7ca)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:40.492325 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:40.540158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a5d87b148964841977ddba4f9b9e7ca'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:40.540762 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a5d87b148964841977ddba4f9b9e7ca'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:40.547822 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a5d87b148964841977ddba4f9b9e7ca'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:40.548554 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a5d87b148964841977ddba4f9b9e7ca'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:40.560737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a5d87b148964841977ddba4f9b9e7ca'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:40.573175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1346/2684] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:13:40 2022] DELETE /identity/v3/users/6a5d87b148964841977ddba4f9b9e7ca => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:40.578354 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f67a9-1162-4adb-9891-b6064bb48811 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:40.578644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f67a9-1162-4adb-9891-b6064bb48811 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:40.578945 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f67a9-1162-4adb-9891-b6064bb48811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:40.596317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-486f67a9-1162-4adb-9891-b6064bb48811 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:40.597545 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-486f67a9-1162-4adb-9891-b6064bb48811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:40.654772 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1339/2685] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:40.664776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:40.666253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:40.725291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a70f6d-c760-4e24-a790-78ac093caff5 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:40.729101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3800aaf-171d-4183-9548-286736ac5834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:40.729751 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3800aaf-171d-4183-9548-286736ac5834 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:40.730037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3800aaf-171d-4183-9548-286736ac5834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:40.730315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3800aaf-171d-4183-9548-286736ac5834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:40.740556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3800aaf-171d-4183-9548-286736ac5834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:40.742006 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3800aaf-171d-4183-9548-286736ac5834 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:40.752125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1347/2686] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:41.018339 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f67a9-1162-4adb-9891-b6064bb48811 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:41.019792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-486f67a9-1162-4adb-9891-b6064bb48811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:41.032194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f67a9-1162-4adb-9891-b6064bb48811 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:41.037051 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:41.037918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.038195 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.038607 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] PATH_INFO: `/v3/projects/64c0ca427b564e258fb670f0b9ce6cb4` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.039795 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c0ca427b564e258fb670f0b9ce6cb4)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:41.041743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:41.089282 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c0ca427b564e258fb670f0b9ce6cb4', 'request_id': 'req-f396911a-5903-446c-a387-5c1b560b8a10'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:41.095851 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c0ca427b564e258fb670f0b9ce6cb4', 'request_id': 'req-f396911a-5903-446c-a387-5c1b560b8a10'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:41.103279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c0ca427b564e258fb670f0b9ce6cb4', 'request_id': 'req-f396911a-5903-446c-a387-5c1b560b8a10'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:41.114792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] The token cache is being invalidate because project 64c0ca427b564e258fb670f0b9ce6cb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:13:41.115081 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:41.115703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f396911a-5903-446c-a387-5c1b560b8a10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:41.117262 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1340/2687] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:13:41 2022] DELETE /identity/v3/projects/64c0ca427b564e258fb670f0b9ce6cb4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:41.122036 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d18de3-3757-4f0f-8b65-cf11cd1eba91 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.122449 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d18de3-3757-4f0f-8b65-cf11cd1eba91 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.122820 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d18de3-3757-4f0f-8b65-cf11cd1eba91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.140381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-d4d18de3-3757-4f0f-8b65-cf11cd1eba91 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:41.141623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d4d18de3-3757-4f0f-8b65-cf11cd1eba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:41.158874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:41.203849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1348/2688] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:41.211526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016f3ce-e6d5-4c6b-b64c-d7dc32021531 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.211896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016f3ce-e6d5-4c6b-b64c-d7dc32021531 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.212084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016f3ce-e6d5-4c6b-b64c-d7dc32021531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.231140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2016f3ce-e6d5-4c6b-b64c-d7dc32021531 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:41.232969 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2016f3ce-e6d5-4c6b-b64c-d7dc32021531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:41.240784 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1341/2689] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:41.244885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d18de3-3757-4f0f-8b65-cf11cd1eba91 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:41.254403 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d18de3-3757-4f0f-8b65-cf11cd1eba91 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:41.259056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a84f3-0a1b-4fdf-b818-7530f8520881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:41.259702 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a84f3-0a1b-4fdf-b818-7530f8520881 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.259985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a84f3-0a1b-4fdf-b818-7530f8520881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.260259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a84f3-0a1b-4fdf-b818-7530f8520881 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.261024 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a84f3-0a1b-4fdf-b818-7530f8520881 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:13:41.261330 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a84f3-0a1b-4fdf-b818-7530f8520881 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:41.262934 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a84f3-0a1b-4fdf-b818-7530f8520881 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:41.274360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1349/2690] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:13:41.279173 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016f3ce-e6d5-4c6b-b64c-d7dc32021531 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:41.288487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016f3ce-e6d5-4c6b-b64c-d7dc32021531 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:41.293293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb7f79-6ed4-463e-8d08-6a38f8161210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:41.293903 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb7f79-6ed4-463e-8d08-6a38f8161210 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.294779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb7f79-6ed4-463e-8d08-6a38f8161210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.295054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb7f79-6ed4-463e-8d08-6a38f8161210 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.295743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb7f79-6ed4-463e-8d08-6a38f8161210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:41.297643 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb7f79-6ed4-463e-8d08-6a38f8161210 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:41.322213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1342/2691] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:13:41 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:13:41.326335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a84f3-0a1b-4fdf-b818-7530f8520881 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:41.336122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a84f3-0a1b-4fdf-b818-7530f8520881 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:41.391045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf381-7397-46f8-b9db-99c7f66889c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:41.391709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf381-7397-46f8-b9db-99c7f66889c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.391994 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf381-7397-46f8-b9db-99c7f66889c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.392273 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf381-7397-46f8-b9db-99c7f66889c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.393039 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf381-7397-46f8-b9db-99c7f66889c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:41.394957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf381-7397-46f8-b9db-99c7f66889c7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:41.415152 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1350/2692] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:13:41 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:13:41.420053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb7f79-6ed4-463e-8d08-6a38f8161210 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:41.429902 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb7f79-6ed4-463e-8d08-6a38f8161210 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:41.434425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945ee90-d5c5-4dd7-b368-71ec10fc01d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:41.435183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945ee90-d5c5-4dd7-b368-71ec10fc01d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.435452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945ee90-d5c5-4dd7-b368-71ec10fc01d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.435729 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945ee90-d5c5-4dd7-b368-71ec10fc01d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.436483 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945ee90-d5c5-4dd7-b368-71ec10fc01d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:41.438074 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945ee90-d5c5-4dd7-b368-71ec10fc01d5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:41.448213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1343/2693] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:13:41.452500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf381-7397-46f8-b9db-99c7f66889c7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:41.462306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf381-7397-46f8-b9db-99c7f66889c7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:41.467470 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8e5da-1067-461e-84a6-575756198e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:41.468122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8e5da-1067-461e-84a6-575756198e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.468427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8e5da-1067-461e-84a6-575756198e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.468701 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8e5da-1067-461e-84a6-575756198e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.469520 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8e5da-1067-461e-84a6-575756198e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:41.471131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8e5da-1067-461e-84a6-575756198e2b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:41.480924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1351/2694] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:13:41.485369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945ee90-d5c5-4dd7-b368-71ec10fc01d5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:41.495043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945ee90-d5c5-4dd7-b368-71ec10fc01d5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:41.500245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb660e2-f3a8-4d0f-aea6-a714e2d65d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:41.501138 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb660e2-f3a8-4d0f-aea6-a714e2d65d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.501622 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb660e2-f3a8-4d0f-aea6-a714e2d65d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.501747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb660e2-f3a8-4d0f-aea6-a714e2d65d22 admin admin] PATH_INFO: `/v3/projects/596b3a1699994f3ba33696a4a7098a1f/users/9afe16fc4bd544d4b0010037fb1812ad/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.516285 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb660e2-f3a8-4d0f-aea6-a714e2d65d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=596b3a1699994f3ba33696a4a7098a1f, user_id=9afe16fc4bd544d4b0010037fb1812ad, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:41.518949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb660e2-f3a8-4d0f-aea6-a714e2d65d22 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:41.531671 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1344/2695] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:13:41 2022] PUT /identity/v3/projects/596b3a1699994f3ba33696a4a7098a1f/users/9afe16fc4bd544d4b0010037fb1812ad/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:41.599692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e67da-f594-41ca-83a0-bdcb17c44af9 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.599890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e67da-f594-41ca-83a0-bdcb17c44af9 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.600154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e67da-f594-41ca-83a0-bdcb17c44af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.628469 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-7d8e67da-f594-41ca-83a0-bdcb17c44af9 None None] MFA Rules not processed for user `9afe16fc4bd544d4b0010037fb1812ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:41.629672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-7d8e67da-f594-41ca-83a0-bdcb17c44af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:41.693569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1352/2696] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:41.704555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb660e2-f3a8-4d0f-aea6-a714e2d65d22 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:41.706288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7fb660e2-f3a8-4d0f-aea6-a714e2d65d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:41.728134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:41.765635 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb660e2-f3a8-4d0f-aea6-a714e2d65d22 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:41.769304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141999d8-1b2b-4c33-8926-3d80b14e274b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:41.769943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141999d8-1b2b-4c33-8926-3d80b14e274b service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:41.770222 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141999d8-1b2b-4c33-8926-3d80b14e274b service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:41.770495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141999d8-1b2b-4c33-8926-3d80b14e274b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:41.779030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141999d8-1b2b-4c33-8926-3d80b14e274b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:41.780676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141999d8-1b2b-4c33-8926-3d80b14e274b service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:41.789948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1345/2697] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:13:51.506476 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e67da-f594-41ca-83a0-bdcb17c44af9 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:51.526361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:13:51.570675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e67da-f594-41ca-83a0-bdcb17c44af9 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:51.575348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:51.576220 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:51.576537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:51.576816 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] PATH_INFO: `/v3/users/9afe16fc4bd544d4b0010037fb1812ad` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:51.578669 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9afe16fc4bd544d4b0010037fb1812ad)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:51.580627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:51.631929 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9afe16fc4bd544d4b0010037fb1812ad'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:51.632601 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9afe16fc4bd544d4b0010037fb1812ad'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:51.639433 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9afe16fc4bd544d4b0010037fb1812ad'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:51.640951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9afe16fc4bd544d4b0010037fb1812ad'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:51.652612 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9afe16fc4bd544d4b0010037fb1812ad'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:51.665618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1353/2698] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:13:51 2022] DELETE /identity/v3/users/9afe16fc4bd544d4b0010037fb1812ad => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:13:51.670358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dac44-83fa-4f30-8ea1-f421633d0cdf None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:51.670905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dac44-83fa-4f30-8ea1-f421633d0cdf None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:51.671083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dac44-83fa-4f30-8ea1-f421633d0cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:51.689053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2a8dac44-83fa-4f30-8ea1-f421633d0cdf None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:13:51.690203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2a8dac44-83fa-4f30-8ea1-f421633d0cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:51.748579 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1346/2699] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:13:51.758784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:51.760188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:51.818628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc48c4c-79bb-499e-a332-a7a96b460cc0 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:51.822283 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0e929-5a64-4927-8ff1-c9470379d13c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:51.822923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0e929-5a64-4927-8ff1-c9470379d13c service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:51.823208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0e929-5a64-4927-8ff1-c9470379d13c service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:51.823482 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0e929-5a64-4927-8ff1-c9470379d13c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:51.832910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0e929-5a64-4927-8ff1-c9470379d13c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:51.834352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0e929-5a64-4927-8ff1-c9470379d13c service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:51.844219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1354/2700] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:13:52.114065 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dac44-83fa-4f30-8ea1-f421633d0cdf None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:13:52.115516 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2a8dac44-83fa-4f30-8ea1-f421633d0cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:13:52.130005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dac44-83fa-4f30-8ea1-f421633d0cdf None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:13:52.134957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:13:52.135917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:13:52.136201 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:13:52.136508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] PATH_INFO: `/v3/projects/596b3a1699994f3ba33696a4a7098a1f` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:13:52.137699 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=596b3a1699994f3ba33696a4a7098a1f)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:13:52.140146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:13:52.186982 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '596b3a1699994f3ba33696a4a7098a1f', 'request_id': 'req-94fafe45-48fc-4cfe-8281-be29607846d3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:52.193103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '596b3a1699994f3ba33696a4a7098a1f', 'request_id': 'req-94fafe45-48fc-4cfe-8281-be29607846d3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:52.200070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '596b3a1699994f3ba33696a4a7098a1f', 'request_id': 'req-94fafe45-48fc-4cfe-8281-be29607846d3'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:52.211278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] The token cache is being invalidate because project 596b3a1699994f3ba33696a4a7098a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:13:52.211572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:52.212194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-94fafe45-48fc-4cfe-8281-be29607846d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:13:52.213743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1347/2701] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:13:52 2022] DELETE /identity/v3/projects/596b3a1699994f3ba33696a4a7098a1f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:14:01.839553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:14:02.208221 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:14:13.543230 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f026fdb-6637-4117-a41f-97d423cfce5d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:13.543765 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f026fdb-6637-4117-a41f-97d423cfce5d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:13.543946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f026fdb-6637-4117-a41f-97d423cfce5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:13.562069 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-8f026fdb-6637-4117-a41f-97d423cfce5d None None] MFA Rules not processed for user `92de4ab63f6447eabefe34f65c71d918`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:14:13.563316 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8f026fdb-6637-4117-a41f-97d423cfce5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:13.621870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1355/2702] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:14:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:14:17.256291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456b88-e07a-485d-a582-aa0fb210b4f5 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:17.256857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456b88-e07a-485d-a582-aa0fb210b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:17.256959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456b88-e07a-485d-a582-aa0fb210b4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:17.278359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-2b456b88-e07a-485d-a582-aa0fb210b4f5 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:14:17.279569 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2b456b88-e07a-485d-a582-aa0fb210b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:17.341840 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1348/2703] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jun 8 02:14:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:14:17.390317 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f026fdb-6637-4117-a41f-97d423cfce5d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:17.392269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8f026fdb-6637-4117-a41f-97d423cfce5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:17.475142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f026fdb-6637-4117-a41f-97d423cfce5d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:17.481478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73b0c7-8950-4c9c-b003-b3837f4afb18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:17.482319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73b0c7-8950-4c9c-b003-b3837f4afb18 service swift] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:17.482688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73b0c7-8950-4c9c-b003-b3837f4afb18 service swift] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:17.483054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73b0c7-8950-4c9c-b003-b3837f4afb18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:17.493515 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73b0c7-8950-4c9c-b003-b3837f4afb18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:17.495804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73b0c7-8950-4c9c-b003-b3837f4afb18 service swift] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:17.505417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1356/2704] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:14:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:14:24.492882 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b456b88-e07a-485d-a582-aa0fb210b4f5 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:24.494343 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2b456b88-e07a-485d-a582-aa0fb210b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:24.570136 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b456b88-e07a-485d-a582-aa0fb210b4f5 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:24.574687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:24.575567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:24.575853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:24.576129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] PATH_INFO: `/v3/users/2f0ae440fa644f1296d2b474c5335ffc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:24.577388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f0ae440fa644f1296d2b474c5335ffc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:24.580139 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:24.627880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f0ae440fa644f1296d2b474c5335ffc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:24.628611 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f0ae440fa644f1296d2b474c5335ffc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:24.635475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f0ae440fa644f1296d2b474c5335ffc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:24.636174 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f0ae440fa644f1296d2b474c5335ffc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:24.644572 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f0ae440fa644f1296d2b474c5335ffc'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:24.662497 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1349/2705] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:14:24 2022] DELETE /identity/v3/users/2f0ae440fa644f1296d2b474c5335ffc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:14:24.834660 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73b0c7-8950-4c9c-b003-b3837f4afb18 service swift] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:24.859275 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73b0c7-8950-4c9c-b003-b3837f4afb18 service swift] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:24.926975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:24.927842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:24.928130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:24.928514 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] PATH_INFO: `/v3/projects/3b0724451ec94fdbb4f2422c44245b53` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:24.929622 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0724451ec94fdbb4f2422c44245b53)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:24.931570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:24.970860 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:14:24.980480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0724451ec94fdbb4f2422c44245b53', 'request_id': 'req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:24.987177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0724451ec94fdbb4f2422c44245b53', 'request_id': 'req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:24.993582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0724451ec94fdbb4f2422c44245b53', 'request_id': 'req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.006280 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] The token cache is being invalidate because project 3b0724451ec94fdbb4f2422c44245b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:14:25.006567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.007200 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.008787 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1357/2706] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:14:24 2022] DELETE /identity/v3/projects/3b0724451ec94fdbb4f2422c44245b53 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:14:25.012771 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:25.014413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:25.028211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4e430-ef4c-4088-95fe-8f8b2cee94f1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:25.049233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:14:25.088461 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:25.089347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.089630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.090417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] PATH_INFO: `/v3/users/960a45d3bb3c4c2cbec522cd11d70d90` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.091210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=960a45d3bb3c4c2cbec522cd11d70d90)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:25.093179 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:25.148783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '960a45d3bb3c4c2cbec522cd11d70d90'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.149473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '960a45d3bb3c4c2cbec522cd11d70d90'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.156219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '960a45d3bb3c4c2cbec522cd11d70d90'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.156885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '960a45d3bb3c4c2cbec522cd11d70d90'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.165161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8698420e-2439-42ea-8941-fdb5c3f63a36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '960a45d3bb3c4c2cbec522cd11d70d90'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.182325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1350/2707] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:14:25 2022] DELETE /identity/v3/users/960a45d3bb3c4c2cbec522cd11d70d90 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:14:25.355276 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:25.356896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:25.371322 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a3fe7-6945-4002-82eb-ef276bfaffd6 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:25.430498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:25.431364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.431650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.431927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] PATH_INFO: `/v3/projects/3424f51f284c42338dc4295f2860a4a4` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.433160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3424f51f284c42338dc4295f2860a4a4)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:25.435106 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:25.482135 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3424f51f284c42338dc4295f2860a4a4', 'request_id': 'req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.487921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3424f51f284c42338dc4295f2860a4a4', 'request_id': 'req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.494155 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3424f51f284c42338dc4295f2860a4a4', 'request_id': 'req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.507394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] The token cache is being invalidate because project 3424f51f284c42338dc4295f2860a4a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:14:25.507683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.508295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-88a96b30-6c2b-4b62-8a6c-57f5f94504a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:14:25.509849 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1358/2708] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:14:25 2022] DELETE /identity/v3/projects/3424f51f284c42338dc4295f2860a4a4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:14:25.514967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7eef0-4c8e-443a-b2f5-77c5609e76aa None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.515458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7eef0-4c8e-443a-b2f5-77c5609e76aa None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.515556 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7eef0-4c8e-443a-b2f5-77c5609e76aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.533672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-a8c7eef0-4c8e-443a-b2f5-77c5609e76aa None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:14:25.534880 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-a8c7eef0-4c8e-443a-b2f5-77c5609e76aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:25.594123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1351/2709] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:14:25.601868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b91915-cab6-4488-b253-2dee9e3737c4 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.602160 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b91915-cab6-4488-b253-2dee9e3737c4 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.602438 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b91915-cab6-4488-b253-2dee9e3737c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.621728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-f1b91915-cab6-4488-b253-2dee9e3737c4 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:14:25.622946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f1b91915-cab6-4488-b253-2dee9e3737c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:25.631697 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1359/2710] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:14:25.635879 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7eef0-4c8e-443a-b2f5-77c5609e76aa None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:25.645895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7eef0-4c8e-443a-b2f5-77c5609e76aa None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:25.650591 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f279ec-5f37-4568-87db-1de89381d81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:25.651245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f279ec-5f37-4568-87db-1de89381d81c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.651525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f279ec-5f37-4568-87db-1de89381d81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.651798 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f279ec-5f37-4568-87db-1de89381d81c admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.652623 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f279ec-5f37-4568-87db-1de89381d81c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:14:25.652929 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f279ec-5f37-4568-87db-1de89381d81c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:25.654551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f279ec-5f37-4568-87db-1de89381d81c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:25.666972 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1352/2711] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:14:25.671153 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b91915-cab6-4488-b253-2dee9e3737c4 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:25.681156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b91915-cab6-4488-b253-2dee9e3737c4 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:25.685773 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f514d41-eccf-4218-bc0b-d75dafe736a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:25.686418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f514d41-eccf-4218-bc0b-d75dafe736a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.686700 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f514d41-eccf-4218-bc0b-d75dafe736a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.686982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f514d41-eccf-4218-bc0b-d75dafe736a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.687674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f514d41-eccf-4218-bc0b-d75dafe736a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:25.691253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f514d41-eccf-4218-bc0b-d75dafe736a0 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:25.718829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1360/2712] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:14:25 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:14:25.720092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f279ec-5f37-4568-87db-1de89381d81c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:25.730036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f279ec-5f37-4568-87db-1de89381d81c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:25.759345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:14:25.789423 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef62a8f-b4bd-4648-8f57-fb21b3aac638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:25.790123 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef62a8f-b4bd-4648-8f57-fb21b3aac638 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.790406 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef62a8f-b4bd-4648-8f57-fb21b3aac638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.790688 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef62a8f-b4bd-4648-8f57-fb21b3aac638 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.791372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef62a8f-b4bd-4648-8f57-fb21b3aac638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:25.793392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef62a8f-b4bd-4648-8f57-fb21b3aac638 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:25.813573 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1353/2713] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:14:25 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:14:25.817609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f514d41-eccf-4218-bc0b-d75dafe736a0 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:25.827832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f514d41-eccf-4218-bc0b-d75dafe736a0 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:25.833303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2a190-65fd-4b5f-96ec-d63f4e2b6f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:25.833956 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2a190-65fd-4b5f-96ec-d63f4e2b6f16 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.834244 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2a190-65fd-4b5f-96ec-d63f4e2b6f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.834526 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2a190-65fd-4b5f-96ec-d63f4e2b6f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.835255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2a190-65fd-4b5f-96ec-d63f4e2b6f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:25.836891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2a190-65fd-4b5f-96ec-d63f4e2b6f16 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:25.847912 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1361/2714] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:14:25.851770 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef62a8f-b4bd-4648-8f57-fb21b3aac638 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:25.861883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef62a8f-b4bd-4648-8f57-fb21b3aac638 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:25.866503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079531a5-bf4e-4591-bd57-9105451a0764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:25.867154 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079531a5-bf4e-4591-bd57-9105451a0764 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.867436 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079531a5-bf4e-4591-bd57-9105451a0764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.867714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079531a5-bf4e-4591-bd57-9105451a0764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.868464 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079531a5-bf4e-4591-bd57-9105451a0764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:25.870067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079531a5-bf4e-4591-bd57-9105451a0764 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:25.880228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1354/2715] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:14:25.884133 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2a190-65fd-4b5f-96ec-d63f4e2b6f16 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:25.894562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2a190-65fd-4b5f-96ec-d63f4e2b6f16 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:25.899417 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057fb94-23e8-482f-9f5e-fbd09a8cce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:25.900289 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057fb94-23e8-482f-9f5e-fbd09a8cce63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.900611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057fb94-23e8-482f-9f5e-fbd09a8cce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.900894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057fb94-23e8-482f-9f5e-fbd09a8cce63 admin admin] PATH_INFO: `/v3/projects/079027b24aba4a179f99de9f6fe944b8/users/89e6c19fcc9f490c890fc4d6350273a0/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.913818 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:14:25.919346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057fb94-23e8-482f-9f5e-fbd09a8cce63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=079027b24aba4a179f99de9f6fe944b8, user_id=89e6c19fcc9f490c890fc4d6350273a0, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:25.922394 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057fb94-23e8-482f-9f5e-fbd09a8cce63 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:25.938044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1362/2716] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:14:25 2022] PUT /identity/v3/projects/079027b24aba4a179f99de9f6fe944b8/users/89e6c19fcc9f490c890fc4d6350273a0/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:14:25.943829 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf241ad-d838-48d7-a380-5fff7fd05a61 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:25.944199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf241ad-d838-48d7-a380-5fff7fd05a61 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:25.944514 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf241ad-d838-48d7-a380-5fff7fd05a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:25.962108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-ddf241ad-d838-48d7-a380-5fff7fd05a61 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:14:25.963302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-ddf241ad-d838-48d7-a380-5fff7fd05a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:26.025099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1355/2717] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:14:26.036941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057fb94-23e8-482f-9f5e-fbd09a8cce63 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:26.038632 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-0057fb94-23e8-482f-9f5e-fbd09a8cce63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:26.102377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057fb94-23e8-482f-9f5e-fbd09a8cce63 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:26.106113 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402d38d-e01a-4549-8abb-3439a3e4de23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:26.106808 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402d38d-e01a-4549-8abb-3439a3e4de23 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:26.107167 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402d38d-e01a-4549-8abb-3439a3e4de23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:26.107445 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402d38d-e01a-4549-8abb-3439a3e4de23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:26.118271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402d38d-e01a-4549-8abb-3439a3e4de23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:26.120064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402d38d-e01a-4549-8abb-3439a3e4de23 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:26.131312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1363/2718] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:14:31.308897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f183f3-bf89-4aae-851d-a1c3facf04f7 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:31.309459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f183f3-bf89-4aae-851d-a1c3facf04f7 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:31.309459 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f183f3-bf89-4aae-851d-a1c3facf04f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:31.342109 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-53f183f3-bf89-4aae-851d-a1c3facf04f7 None None] MFA Rules not processed for user `89e6c19fcc9f490c890fc4d6350273a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:14:31.343321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-53f183f3-bf89-4aae-851d-a1c3facf04f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:31.416130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1356/2719] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:14:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:14:31.420874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402d38d-e01a-4549-8abb-3439a3e4de23 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:31.432933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402d38d-e01a-4549-8abb-3439a3e4de23 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:31.437921 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578f42b-5304-4853-88c8-3626f4da0402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:31.439539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578f42b-5304-4853-88c8-3626f4da0402 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:31.440021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578f42b-5304-4853-88c8-3626f4da0402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:31.440506 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578f42b-5304-4853-88c8-3626f4da0402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:31.441297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578f42b-5304-4853-88c8-3626f4da0402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:31.443288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578f42b-5304-4853-88c8-3626f4da0402 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:31.471661 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1364/2720] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:14:31 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:14:31.475718 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f183f3-bf89-4aae-851d-a1c3facf04f7 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:31.487470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f183f3-bf89-4aae-851d-a1c3facf04f7 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:31.529887 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:14:31.551621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5c9d7-b513-44df-84e6-22ec2955cef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:31.552304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5c9d7-b513-44df-84e6-22ec2955cef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:31.552663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5c9d7-b513-44df-84e6-22ec2955cef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:31.552938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5c9d7-b513-44df-84e6-22ec2955cef6 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:31.553634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5c9d7-b513-44df-84e6-22ec2955cef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:31.555574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5c9d7-b513-44df-84e6-22ec2955cef6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:31.577355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1357/2721] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:14:31 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:14:31.581980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578f42b-5304-4853-88c8-3626f4da0402 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:31.592814 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578f42b-5304-4853-88c8-3626f4da0402 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:31.597628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d66822-8bc6-4d9c-9c79-0099550b2257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:31.598310 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d66822-8bc6-4d9c-9c79-0099550b2257 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:31.598597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d66822-8bc6-4d9c-9c79-0099550b2257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:31.598879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d66822-8bc6-4d9c-9c79-0099550b2257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:31.599609 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d66822-8bc6-4d9c-9c79-0099550b2257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:31.601271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d66822-8bc6-4d9c-9c79-0099550b2257 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:31.613028 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1365/2722] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:14:31.617231 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5c9d7-b513-44df-84e6-22ec2955cef6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:31.628764 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5c9d7-b513-44df-84e6-22ec2955cef6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:31.633596 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0d628-aa73-43c2-82a6-20df4fbad562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:31.634502 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0d628-aa73-43c2-82a6-20df4fbad562 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:31.634905 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0d628-aa73-43c2-82a6-20df4fbad562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:31.635186 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0d628-aa73-43c2-82a6-20df4fbad562 admin admin] PATH_INFO: `/v3/projects/d56f3fbee70c4c9d98af73d13cbd9a0e/users/08b187e774f745418614c327e14d8c31/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:31.650873 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0d628-aa73-43c2-82a6-20df4fbad562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56f3fbee70c4c9d98af73d13cbd9a0e, user_id=08b187e774f745418614c327e14d8c31, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:31.653728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0d628-aa73-43c2-82a6-20df4fbad562 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:31.669188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1358/2723] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:14:31 2022] PUT /identity/v3/projects/d56f3fbee70c4c9d98af73d13cbd9a0e/users/08b187e774f745418614c327e14d8c31/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:14:31.673605 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d66822-8bc6-4d9c-9c79-0099550b2257 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:31.683964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d66822-8bc6-4d9c-9c79-0099550b2257 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:31.742919 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9f055-f547-4d0f-aff2-30fc051915d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:31.744074 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9f055-f547-4d0f-aff2-30fc051915d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:31.744607 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9f055-f547-4d0f-aff2-30fc051915d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:31.745088 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9f055-f547-4d0f-aff2-30fc051915d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:31.746387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9f055-f547-4d0f-aff2-30fc051915d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:31.748242 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9f055-f547-4d0f-aff2-30fc051915d3 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:31.761313 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1366/2724] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:14:31.766903 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0d628-aa73-43c2-82a6-20df4fbad562 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:31.782147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0d628-aa73-43c2-82a6-20df4fbad562 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:31.789239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f24dca-d510-45ed-872a-6d0269cd894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:31.790613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24dca-d510-45ed-872a-6d0269cd894f admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:31.791046 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24dca-d510-45ed-872a-6d0269cd894f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:31.791470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24dca-d510-45ed-872a-6d0269cd894f admin admin] PATH_INFO: `/v3/projects/d56f3fbee70c4c9d98af73d13cbd9a0e/users/08b187e774f745418614c327e14d8c31/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:31.794444 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f24dca-d510-45ed-872a-6d0269cd894f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56f3fbee70c4c9d98af73d13cbd9a0e, user_id=08b187e774f745418614c327e14d8c31, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:31.799392 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f24dca-d510-45ed-872a-6d0269cd894f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:31.812150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1359/2725] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:14:31 2022] PUT /identity/v3/projects/d56f3fbee70c4c9d98af73d13cbd9a0e/users/08b187e774f745418614c327e14d8c31/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:14:36.660570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fa568-8664-49a0-ac26-34e9c0aa978c None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:36.661390 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fa568-8664-49a0-ac26-34e9c0aa978c None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:36.661505 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fa568-8664-49a0-ac26-34e9c0aa978c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:36.695725 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-986fa568-8664-49a0-ac26-34e9c0aa978c None None] MFA Rules not processed for user `08b187e774f745418614c327e14d8c31`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:14:36.696986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-986fa568-8664-49a0-ac26-34e9c0aa978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:36.757185 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:14:36.781361 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1367/2726] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:14:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:14:36.800881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f24dca-d510-45ed-872a-6d0269cd894f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:14:36.807952 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d0f24dca-d510-45ed-872a-6d0269cd894f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:36.900442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f24dca-d510-45ed-872a-6d0269cd894f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:14:36.905983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf35c22-f2a8-4122-bbf0-bca01c1a4ca3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:14:36.906737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf35c22-f2a8-4122-bbf0-bca01c1a4ca3 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:36.907650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf35c22-f2a8-4122-bbf0-bca01c1a4ca3 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:36.907996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf35c22-f2a8-4122-bbf0-bca01c1a4ca3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:36.968568 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf35c22-f2a8-4122-bbf0-bca01c1a4ca3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:14:36.970278 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf35c22-f2a8-4122-bbf0-bca01c1a4ca3 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:14:36.979408 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1360/2727] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:14:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:14:46.778215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=8 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:14:46.975312 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:14:54.225336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edfa38-7c84-4754-abf1-094ddde10b3a None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:14:54.225858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edfa38-7c84-4754-abf1-094ddde10b3a None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:14:54.225965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edfa38-7c84-4754-abf1-094ddde10b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:14:54.243958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-80edfa38-7c84-4754-abf1-094ddde10b3a None None] MFA Rules not processed for user `89e6c19fcc9f490c890fc4d6350273a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:14:54.245177 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-80edfa38-7c84-4754-abf1-094ddde10b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:14:54.251874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1368/2728] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:14:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:15:04.238584 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:15:13.449472 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf35c22-f2a8-4122-bbf0-bca01c1a4ca3 service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:15:13.525257 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf35c22-f2a8-4122-bbf0-bca01c1a4ca3 service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:15:13.528846 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdd29c-9ede-43f5-86fb-129fd8089b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:15:13.529500 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdd29c-9ede-43f5-86fb-129fd8089b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:15:13.529781 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdd29c-9ede-43f5-86fb-129fd8089b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:15:13.530054 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdd29c-9ede-43f5-86fb-129fd8089b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:15:13.541020 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdd29c-9ede-43f5-86fb-129fd8089b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:15:13.542239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdd29c-9ede-43f5-86fb-129fd8089b38 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:15:13.553856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1361/2729] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:15:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:15:15.050262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee04043-a0be-4def-8891-a5c23c4569f2 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:15:15.050838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee04043-a0be-4def-8891-a5c23c4569f2 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:15:15.050838 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee04043-a0be-4def-8891-a5c23c4569f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:15:15.069059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-aee04043-a0be-4def-8891-a5c23c4569f2 None None] MFA Rules not processed for user `92de4ab63f6447eabefe34f65c71d918`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:15:15.070498 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-aee04043-a0be-4def-8891-a5c23c4569f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:15:15.127207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1369/2730] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:15:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:15:23.283430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdd29c-9ede-43f5-86fb-129fd8089b38 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:15:23.285520 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-42fdd29c-9ede-43f5-86fb-129fd8089b38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:15:23.297040 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdd29c-9ede-43f5-86fb-129fd8089b38 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:15:23.300811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119930d-0b60-40d9-a648-02a5b44c8195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:15:23.301452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119930d-0b60-40d9-a648-02a5b44c8195 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:15:23.301736 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119930d-0b60-40d9-a648-02a5b44c8195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:15:23.302014 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119930d-0b60-40d9-a648-02a5b44c8195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:15:23.303755 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3119930d-0b60-40d9-a648-02a5b44c8195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:15:23.317093 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119930d-0b60-40d9-a648-02a5b44c8195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:15:23.318504 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119930d-0b60-40d9-a648-02a5b44c8195 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:15:23.329432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1362/2731] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:15:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:15:25.120031 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=25 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:15:25.761338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee04043-a0be-4def-8891-a5c23c4569f2 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:15:25.771757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee04043-a0be-4def-8891-a5c23c4569f2 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:15:25.775303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e9655-b9ca-421e-a42c-ac315f8367a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:15:25.775938 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e9655-b9ca-421e-a42c-ac315f8367a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:15:25.776224 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e9655-b9ca-421e-a42c-ac315f8367a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:15:25.776539 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e9655-b9ca-421e-a42c-ac315f8367a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:15:25.778291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1d9e9655-b9ca-421e-a42c-ac315f8367a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:15:25.793049 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e9655-b9ca-421e-a42c-ac315f8367a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:15:25.794467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e9655-b9ca-421e-a42c-ac315f8367a0 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:15:25.805597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1370/2732] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:15:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:15:33.324538 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=32 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:15:35.801011 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=6 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:16:17.404511 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119930d-0b60-40d9-a648-02a5b44c8195 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:16:17.406062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-3119930d-0b60-40d9-a648-02a5b44c8195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:16:17.424145 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119930d-0b60-40d9-a648-02a5b44c8195 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:16:17.431033 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcaf7e-ba1a-4295-a338-81c4b692e9d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:16:17.431849 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcaf7e-ba1a-4295-a338-81c4b692e9d0 service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:16:17.432144 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcaf7e-ba1a-4295-a338-81c4b692e9d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:16:17.432439 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcaf7e-ba1a-4295-a338-81c4b692e9d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:16:17.434425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-45dcaf7e-ba1a-4295-a338-81c4b692e9d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:16:17.456719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcaf7e-ba1a-4295-a338-81c4b692e9d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:16:17.460226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcaf7e-ba1a-4295-a338-81c4b692e9d0 service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:16:17.473820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1363/2733] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:16:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:16:20.507064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff0e4c-0c5b-4041-99b6-472852e42dc8 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:16:20.507567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff0e4c-0c5b-4041-99b6-472852e42dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:16:20.507682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff0e4c-0c5b-4041-99b6-472852e42dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:16:20.526140 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-baff0e4c-0c5b-4041-99b6-472852e42dc8 None None] MFA Rules not processed for user `92de4ab63f6447eabefe34f65c71d918`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:16:20.527980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-baff0e4c-0c5b-4041-99b6-472852e42dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:16:20.535078 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1371/2734] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:16:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:16:21.155171 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcaf7e-ba1a-4295-a338-81c4b692e9d0 service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:16:21.156630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-45dcaf7e-ba1a-4295-a338-81c4b692e9d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:16:21.231020 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcaf7e-ba1a-4295-a338-81c4b692e9d0 service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:16:21.235957 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bc145-b5c6-4aa0-82af-b23513c30371 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:16:21.236647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bc145-b5c6-4aa0-82af-b23513c30371 service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:16:21.236933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bc145-b5c6-4aa0-82af-b23513c30371 service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:16:21.237235 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bc145-b5c6-4aa0-82af-b23513c30371 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:16:21.239004 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-eb5bc145-b5c6-4aa0-82af-b23513c30371 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:16:21.250797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bc145-b5c6-4aa0-82af-b23513c30371 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:16:21.252369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bc145-b5c6-4aa0-82af-b23513c30371 service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:16:21.262916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1364/2735] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:16:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:16:23.005440 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff0e4c-0c5b-4041-99b6-472852e42dc8 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:16:23.016442 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff0e4c-0c5b-4041-99b6-472852e42dc8 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:16:23.023473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df17d82-58f1-4c69-b7ab-6125f302d835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:16:23.024124 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df17d82-58f1-4c69-b7ab-6125f302d835 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:16:23.024423 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df17d82-58f1-4c69-b7ab-6125f302d835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:16:23.024705 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df17d82-58f1-4c69-b7ab-6125f302d835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:16:23.026492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5df17d82-58f1-4c69-b7ab-6125f302d835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:16:23.036945 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df17d82-58f1-4c69-b7ab-6125f302d835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:16:23.038704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df17d82-58f1-4c69-b7ab-6125f302d835 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:16:23.047465 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1372/2736] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:16:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:16:28.735413 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bc145-b5c6-4aa0-82af-b23513c30371 service placement] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:16:28.750101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bc145-b5c6-4aa0-82af-b23513c30371 service placement] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:16:28.756593 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f319397-3a54-439b-a025-d2dd206dc16c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:16:28.757256 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f319397-3a54-439b-a025-d2dd206dc16c service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:16:28.757540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f319397-3a54-439b-a025-d2dd206dc16c service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:16:28.757819 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f319397-3a54-439b-a025-d2dd206dc16c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:16:28.759605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1f319397-3a54-439b-a025-d2dd206dc16c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:16:28.772510 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f319397-3a54-439b-a025-d2dd206dc16c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:16:28.774303 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f319397-3a54-439b-a025-d2dd206dc16c service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:16:28.785664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1365/2737] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:16:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:16:33.043545 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=9 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:16:38.781613 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=44 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:14.090303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c41ab-559b-4503-863a-55973692b981 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:14.090843 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c41ab-559b-4503-863a-55973692b981 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:14.090843 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c41ab-559b-4503-863a-55973692b981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:14.119411 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-5e8c41ab-559b-4503-863a-55973692b981 None None] MFA Rules not processed for user `89e6c19fcc9f490c890fc4d6350273a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:14.120670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5e8c41ab-559b-4503-863a-55973692b981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:14.145779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1373/2738] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:14.456840 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f319397-3a54-439b-a025-d2dd206dc16c service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:14.493478 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f319397-3a54-439b-a025-d2dd206dc16c service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:14.497933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd29558-c48c-4429-8293-71c7726ef7b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:14.498582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29558-c48c-4429-8293-71c7726ef7b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:14.498867 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29558-c48c-4429-8293-71c7726ef7b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:14.499143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29558-c48c-4429-8293-71c7726ef7b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:14.500949 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2cd29558-c48c-4429-8293-71c7726ef7b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:14.511067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd29558-c48c-4429-8293-71c7726ef7b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:14.512506 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd29558-c48c-4429-8293-71c7726ef7b7 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:14.520406 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1366/2739] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:17:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:17:24.143717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=15 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:24.516812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=18 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:31.856557 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c41ab-559b-4503-863a-55973692b981 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:31.858118 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-5e8c41ab-559b-4503-863a-55973692b981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:31.893779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c41ab-559b-4503-863a-55973692b981 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:31.900852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd39a9-5a87-4b49-a8bc-a2a584a5e42a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:31.902702 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd39a9-5a87-4b49-a8bc-a2a584a5e42a service placement] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:31.902985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd39a9-5a87-4b49-a8bc-a2a584a5e42a service placement] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:31.903259 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd39a9-5a87-4b49-a8bc-a2a584a5e42a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:31.905803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a2cd39a9-5a87-4b49-a8bc-a2a584a5e42a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:31.928356 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd29558-c48c-4429-8293-71c7726ef7b7 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:31.930649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-2cd29558-c48c-4429-8293-71c7726ef7b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:31.931237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd39a9-5a87-4b49-a8bc-a2a584a5e42a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:31.934910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd39a9-5a87-4b49-a8bc-a2a584a5e42a service placement] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:31.955428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1374/2740] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:17:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:17:32.052357 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd29558-c48c-4429-8293-71c7726ef7b7 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:32.057978 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:32.058864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.059148 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:32.059602 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] PATH_INFO: `/v3/users/92de4ab63f6447eabefe34f65c71d918` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:32.060862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92de4ab63f6447eabefe34f65c71d918)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:32.062794 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:32.116728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92de4ab63f6447eabefe34f65c71d918'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:32.117387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92de4ab63f6447eabefe34f65c71d918'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:32.124346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92de4ab63f6447eabefe34f65c71d918'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:32.125008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92de4ab63f6447eabefe34f65c71d918'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:32.134043 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-875651fa-af94-4dff-92fd-f2d3a55ed430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92de4ab63f6447eabefe34f65c71d918'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:32.157251 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1367/2741] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:17:31 2022] DELETE /identity/v3/users/92de4ab63f6447eabefe34f65c71d918 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:32.333070 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd39a9-5a87-4b49-a8bc-a2a584a5e42a service placement] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:32.348905 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd39a9-5a87-4b49-a8bc-a2a584a5e42a service placement] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:32.411428 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:32.412295 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.412611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:32.412887 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] PATH_INFO: `/v3/projects/879e3eacb3bc4b69891677c86bbd5de3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:32.414328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=879e3eacb3bc4b69891677c86bbd5de3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:32.416233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:32.465219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879e3eacb3bc4b69891677c86bbd5de3', 'request_id': 'req-62cdbc3b-beef-4628-9c38-db740b77ca3a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:32.471377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879e3eacb3bc4b69891677c86bbd5de3', 'request_id': 'req-62cdbc3b-beef-4628-9c38-db740b77ca3a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:32.478335 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879e3eacb3bc4b69891677c86bbd5de3', 'request_id': 'req-62cdbc3b-beef-4628-9c38-db740b77ca3a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:32.492062 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] The token cache is being invalidate because project 879e3eacb3bc4b69891677c86bbd5de3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:17:32.492356 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:32.493015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-62cdbc3b-beef-4628-9c38-db740b77ca3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:32.494553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1375/2742] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:17:32 2022] DELETE /identity/v3/projects/879e3eacb3bc4b69891677c86bbd5de3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:32.499996 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb25d3-0bc2-475d-910c-f1c6c94783eb None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.500390 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb25d3-0bc2-475d-910c-f1c6c94783eb None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:32.500800 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb25d3-0bc2-475d-910c-f1c6c94783eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:32.561056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-9acb25d3-0bc2-475d-910c-f1c6c94783eb None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:32.562426 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-9acb25d3-0bc2-475d-910c-f1c6c94783eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:32.576689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:32.627621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1368/2743] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:32.635404 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4afee-591e-4f78-b7ca-c9ee2c87c0c4 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.635795 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4afee-591e-4f78-b7ca-c9ee2c87c0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:32.636150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4afee-591e-4f78-b7ca-c9ee2c87c0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:32.655207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-09c4afee-591e-4f78-b7ca-c9ee2c87c0c4 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:32.656571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-09c4afee-591e-4f78-b7ca-c9ee2c87c0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:32.664790 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1376/2744] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:32.669015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb25d3-0bc2-475d-910c-f1c6c94783eb None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:32.678737 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb25d3-0bc2-475d-910c-f1c6c94783eb None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:32.683961 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2db5f6-509d-484b-b370-ec437a56ddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:32.684657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2db5f6-509d-484b-b370-ec437a56ddd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.684946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2db5f6-509d-484b-b370-ec437a56ddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:32.685224 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2db5f6-509d-484b-b370-ec437a56ddd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:32.685944 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2db5f6-509d-484b-b370-ec437a56ddd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:17:32.686252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2db5f6-509d-484b-b370-ec437a56ddd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:32.687869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2db5f6-509d-484b-b370-ec437a56ddd6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:32.704462 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1369/2745] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:17:32.705606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4afee-591e-4f78-b7ca-c9ee2c87c0c4 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:32.715714 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4afee-591e-4f78-b7ca-c9ee2c87c0c4 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:32.720653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896656a6-e8d4-48e4-a687-391ce9f5ab0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:32.721309 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896656a6-e8d4-48e4-a687-391ce9f5ab0d admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.721596 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896656a6-e8d4-48e4-a687-391ce9f5ab0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:32.721876 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896656a6-e8d4-48e4-a687-391ce9f5ab0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:32.722599 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896656a6-e8d4-48e4-a687-391ce9f5ab0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:32.724535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896656a6-e8d4-48e4-a687-391ce9f5ab0d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:32.755008 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1377/2746] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:17:32 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:32.756248 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2db5f6-509d-484b-b370-ec437a56ddd6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:32.766454 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2db5f6-509d-484b-b370-ec437a56ddd6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:32.825738 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd782e9-7ee8-423d-a43f-72b2298e868c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:32.826331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd782e9-7ee8-423d-a43f-72b2298e868c admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.826620 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd782e9-7ee8-423d-a43f-72b2298e868c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:32.826898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd782e9-7ee8-423d-a43f-72b2298e868c admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:32.827583 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd782e9-7ee8-423d-a43f-72b2298e868c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:32.829534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd782e9-7ee8-423d-a43f-72b2298e868c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:32.855715 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1370/2747] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:17:32 2022] POST /identity/v3/users => generated 279 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:32.857044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896656a6-e8d4-48e4-a687-391ce9f5ab0d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:32.867577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896656a6-e8d4-48e4-a687-391ce9f5ab0d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:32.872466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a866-db03-4636-9a0e-15dd305c5fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:32.873159 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a866-db03-4636-9a0e-15dd305c5fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.873447 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a866-db03-4636-9a0e-15dd305c5fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:32.873729 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a866-db03-4636-9a0e-15dd305c5fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:32.874467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911a866-db03-4636-9a0e-15dd305c5fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:32.876096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911a866-db03-4636-9a0e-15dd305c5fd1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:32.887543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1378/2748] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:32.891393 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd782e9-7ee8-423d-a43f-72b2298e868c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:32.901387 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd782e9-7ee8-423d-a43f-72b2298e868c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:32.906181 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a0fec-b643-4555-8fb5-2b51c62764de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:32.906931 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a0fec-b643-4555-8fb5-2b51c62764de admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.907225 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a0fec-b643-4555-8fb5-2b51c62764de admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:32.907486 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a0fec-b643-4555-8fb5-2b51c62764de admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:32.908306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a0fec-b643-4555-8fb5-2b51c62764de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:32.909865 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a0fec-b643-4555-8fb5-2b51c62764de admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:32.932230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1371/2749] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:32.934410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a866-db03-4636-9a0e-15dd305c5fd1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:32.946392 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a866-db03-4636-9a0e-15dd305c5fd1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:32.951502 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5c0a0-2356-4fd4-b2b2-ef5ada8447ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:32.952550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5c0a0-2356-4fd4-b2b2-ef5ada8447ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.952931 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5c0a0-2356-4fd4-b2b2-ef5ada8447ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:32.953422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5c0a0-2356-4fd4-b2b2-ef5ada8447ee admin admin] PATH_INFO: `/v3/projects/4e465b92eb994df7bb357bf9057f19c1/users/ac0be5528dee43399eede13b2d0ac5de/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:32.969955 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5c0a0-2356-4fd4-b2b2-ef5ada8447ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e465b92eb994df7bb357bf9057f19c1, user_id=ac0be5528dee43399eede13b2d0ac5de, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:32.973007 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5c0a0-2356-4fd4-b2b2-ef5ada8447ee admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:32.989104 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1379/2750] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:17:32 2022] PUT /identity/v3/projects/4e465b92eb994df7bb357bf9057f19c1/users/ac0be5528dee43399eede13b2d0ac5de/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:32.999919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930002-7b43-4ad8-9b19-e91e26780e9d None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:32.999919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930002-7b43-4ad8-9b19-e91e26780e9d None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:33.000247 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87930002-7b43-4ad8-9b19-e91e26780e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:33.048442 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-87930002-7b43-4ad8-9b19-e91e26780e9d None None] MFA Rules not processed for user `ac0be5528dee43399eede13b2d0ac5de`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:33.049848 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-87930002-7b43-4ad8-9b19-e91e26780e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:33.125232 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1372/2751] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:32 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:33.126684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5c0a0-2356-4fd4-b2b2-ef5ada8447ee admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:33.136333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:33.196207 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5c0a0-2356-4fd4-b2b2-ef5ada8447ee admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:33.201112 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a3dec-8cf7-4005-aa4e-242872c4baee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:33.201779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a3dec-8cf7-4005-aa4e-242872c4baee admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:33.202063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a3dec-8cf7-4005-aa4e-242872c4baee admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:33.202336 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a3dec-8cf7-4005-aa4e-242872c4baee admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:33.203025 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a3dec-8cf7-4005-aa4e-242872c4baee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:33.204978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a3dec-8cf7-4005-aa4e-242872c4baee admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:33.234439 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1380/2752] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:17:33 2022] POST /identity/v3/projects => generated 357 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:33.239024 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87930002-7b43-4ad8-9b19-e91e26780e9d None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:33.250359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87930002-7b43-4ad8-9b19-e91e26780e9d None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:33.262267 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:33.323152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143d858-bf53-46dc-bbf6-b70ebb4ce101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:33.323822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143d858-bf53-46dc-bbf6-b70ebb4ce101 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:33.324104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143d858-bf53-46dc-bbf6-b70ebb4ce101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:33.324375 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143d858-bf53-46dc-bbf6-b70ebb4ce101 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:33.325091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143d858-bf53-46dc-bbf6-b70ebb4ce101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:33.327002 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143d858-bf53-46dc-bbf6-b70ebb4ce101 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:33.349549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1373/2753] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:17:33 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:17:33.353431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a3dec-8cf7-4005-aa4e-242872c4baee admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:33.364578 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a3dec-8cf7-4005-aa4e-242872c4baee admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:33.369427 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12cf6d-f609-46e7-984b-e264ddddc98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:33.370195 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12cf6d-f609-46e7-984b-e264ddddc98a admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:33.370617 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12cf6d-f609-46e7-984b-e264ddddc98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:33.370968 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12cf6d-f609-46e7-984b-e264ddddc98a admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:33.371771 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12cf6d-f609-46e7-984b-e264ddddc98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:33.373640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12cf6d-f609-46e7-984b-e264ddddc98a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:33.388038 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1381/2754] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:33.389345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143d858-bf53-46dc-bbf6-b70ebb4ce101 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:33.399328 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143d858-bf53-46dc-bbf6-b70ebb4ce101 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:33.404762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22275350-5d92-47a3-a879-d625f9ec9101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:33.405647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22275350-5d92-47a3-a879-d625f9ec9101 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:33.405926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22275350-5d92-47a3-a879-d625f9ec9101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:33.406198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22275350-5d92-47a3-a879-d625f9ec9101 admin admin] PATH_INFO: `/v3/projects/b34e45c472eb4f2b9e967053d0ff9a4a/users/743af87b670d4403ab067c27209d8c92/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:33.421933 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22275350-5d92-47a3-a879-d625f9ec9101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34e45c472eb4f2b9e967053d0ff9a4a, user_id=743af87b670d4403ab067c27209d8c92, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:33.424632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22275350-5d92-47a3-a879-d625f9ec9101 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:33.439983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1374/2755] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:17:33 2022] PUT /identity/v3/projects/b34e45c472eb4f2b9e967053d0ff9a4a/users/743af87b670d4403ab067c27209d8c92/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:33.443958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12cf6d-f609-46e7-984b-e264ddddc98a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:33.454149 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12cf6d-f609-46e7-984b-e264ddddc98a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:33.512723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb2b34-e416-4553-ae56-1c132ca41717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:33.513396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb2b34-e416-4553-ae56-1c132ca41717 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:33.513682 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb2b34-e416-4553-ae56-1c132ca41717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:33.513962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb2b34-e416-4553-ae56-1c132ca41717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:33.514693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb2b34-e416-4553-ae56-1c132ca41717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:33.516331 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb2b34-e416-4553-ae56-1c132ca41717 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:33.527122 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1382/2756] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:33.531230 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22275350-5d92-47a3-a879-d625f9ec9101 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:33.545086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22275350-5d92-47a3-a879-d625f9ec9101 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:33.551731 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5238c-8e2e-4ede-9d2f-30f780653b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:33.552869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5238c-8e2e-4ede-9d2f-30f780653b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:33.553157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5238c-8e2e-4ede-9d2f-30f780653b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:33.553432 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5238c-8e2e-4ede-9d2f-30f780653b9d admin admin] PATH_INFO: `/v3/projects/b34e45c472eb4f2b9e967053d0ff9a4a/users/743af87b670d4403ab067c27209d8c92/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:33.555471 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5238c-8e2e-4ede-9d2f-30f780653b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34e45c472eb4f2b9e967053d0ff9a4a, user_id=743af87b670d4403ab067c27209d8c92, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:33.559078 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5238c-8e2e-4ede-9d2f-30f780653b9d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:33.571998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1375/2757] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:17:33 2022] PUT /identity/v3/projects/b34e45c472eb4f2b9e967053d0ff9a4a/users/743af87b670d4403ab067c27209d8c92/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:33.584533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ea932-ee4e-4c6e-8066-9f1c442fa54f None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:33.584933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ea932-ee4e-4c6e-8066-9f1c442fa54f None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:33.585274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ea932-ee4e-4c6e-8066-9f1c442fa54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:33.614767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-740ea932-ee4e-4c6e-8066-9f1c442fa54f None None] MFA Rules not processed for user `743af87b670d4403ab067c27209d8c92`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:33.616094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-740ea932-ee4e-4c6e-8066-9f1c442fa54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:33.689003 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1383/2758] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:33 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:33.700324 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5238c-8e2e-4ede-9d2f-30f780653b9d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:33.702068 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d1a5238c-8e2e-4ede-9d2f-30f780653b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:33.761119 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5238c-8e2e-4ede-9d2f-30f780653b9d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:33.764751 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d784a-1865-4f17-80b5-44dce00fbfe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:33.765403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d784a-1865-4f17-80b5-44dce00fbfe7 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:33.765681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d784a-1865-4f17-80b5-44dce00fbfe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:33.765960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d784a-1865-4f17-80b5-44dce00fbfe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:33.775210 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d784a-1865-4f17-80b5-44dce00fbfe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:33.777304 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d784a-1865-4f17-80b5-44dce00fbfe7 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:33.787167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1376/2759] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:17:33 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:17:33.861792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ea932-ee4e-4c6e-8066-9f1c442fa54f None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:33.866613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:33.872980 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ea932-ee4e-4c6e-8066-9f1c442fa54f None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:33.877455 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69ef06-1bd0-4887-9f58-284c197fbbfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:33.878145 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69ef06-1bd0-4887-9f58-284c197fbbfe service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:33.878432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69ef06-1bd0-4887-9f58-284c197fbbfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:33.878712 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69ef06-1bd0-4887-9f58-284c197fbbfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:33.945130 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69ef06-1bd0-4887-9f58-284c197fbbfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:33.946593 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69ef06-1bd0-4887-9f58-284c197fbbfe service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:33.956236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1384/2760] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:17:33 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:17:43.782561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=63,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:43.951962 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=27 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:48.856085 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d784a-1865-4f17-80b5-44dce00fbfe7 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:48.921942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d784a-1865-4f17-80b5-44dce00fbfe7 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:48.926525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:48.927403 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:48.927687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:48.927967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] PATH_INFO: `/v3/users/ac0be5528dee43399eede13b2d0ac5de` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:48.929203 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0be5528dee43399eede13b2d0ac5de)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:48.931161 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:48.977438 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0be5528dee43399eede13b2d0ac5de'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:48.978087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0be5528dee43399eede13b2d0ac5de'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:48.985503 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0be5528dee43399eede13b2d0ac5de'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:48.986128 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0be5528dee43399eede13b2d0ac5de'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:48.993146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0be5528dee43399eede13b2d0ac5de'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.010335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1377/2761] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:17:48 2022] DELETE /identity/v3/users/ac0be5528dee43399eede13b2d0ac5de => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:49.014776 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb0873-2c2c-4be2-b95d-d514c1c509cc None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:49.015257 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb0873-2c2c-4be2-b95d-d514c1c509cc None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:49.015392 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb0873-2c2c-4be2-b95d-d514c1c509cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:49.032676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-01cb0873-2c2c-4be2-b95d-d514c1c509cc None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:49.033999 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-01cb0873-2c2c-4be2-b95d-d514c1c509cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:49.095805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1385/2762] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:49.105750 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:49.107152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:49.176011 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38e7b9-b9f7-4d2a-8e01-220e6de841e5 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:49.179673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d141b2e-2a61-4a82-b1d1-61b64ffab305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:49.180320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d141b2e-2a61-4a82-b1d1-61b64ffab305 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:49.180639 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d141b2e-2a61-4a82-b1d1-61b64ffab305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:49.180916 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d141b2e-2a61-4a82-b1d1-61b64ffab305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:49.189934 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d141b2e-2a61-4a82-b1d1-61b64ffab305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:49.191380 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d141b2e-2a61-4a82-b1d1-61b64ffab305 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:49.201484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1378/2763] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:17:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:17:49.466154 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb0873-2c2c-4be2-b95d-d514c1c509cc None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:49.467739 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-01cb0873-2c2c-4be2-b95d-d514c1c509cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:49.481852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb0873-2c2c-4be2-b95d-d514c1c509cc None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:49.486535 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:49.487930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:49.488219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:49.488708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] PATH_INFO: `/v3/projects/4e465b92eb994df7bb357bf9057f19c1` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:49.490058 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e465b92eb994df7bb357bf9057f19c1)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:49.492109 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:49.538511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e465b92eb994df7bb357bf9057f19c1', 'request_id': 'req-44c72c8c-f5df-437e-b413-b8f972454e91'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.544334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e465b92eb994df7bb357bf9057f19c1', 'request_id': 'req-44c72c8c-f5df-437e-b413-b8f972454e91'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.550474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e465b92eb994df7bb357bf9057f19c1', 'request_id': 'req-44c72c8c-f5df-437e-b413-b8f972454e91'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.563101 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] The token cache is being invalidate because project 4e465b92eb994df7bb357bf9057f19c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:17:49.563392 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.564009 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.565567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1386/2764] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:17:49 2022] DELETE /identity/v3/projects/4e465b92eb994df7bb357bf9057f19c1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:49.569658 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d141b2e-2a61-4a82-b1d1-61b64ffab305 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:49.579980 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d141b2e-2a61-4a82-b1d1-61b64ffab305 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:49.587543 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:49.641983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:49.642776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:49.643063 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:49.643341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] PATH_INFO: `/v3/users/743af87b670d4403ab067c27209d8c92` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:49.644551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] RBAC: Authorizing `identity:delete_user(user_id=743af87b670d4403ab067c27209d8c92)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:49.646489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:49.692728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743af87b670d4403ab067c27209d8c92'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.693377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743af87b670d4403ab067c27209d8c92'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.700966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743af87b670d4403ab067c27209d8c92'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.701595 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743af87b670d4403ab067c27209d8c92'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.708491 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-858f5093-a745-41b3-b858-1c8be3a4712a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743af87b670d4403ab067c27209d8c92'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:49.725721 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1379/2765] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:17:49 2022] DELETE /identity/v3/users/743af87b670d4403ab067c27209d8c92 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:49.993362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:49.995315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:50.008284 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c72c8c-f5df-437e-b413-b8f972454e91 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.066529 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.067396 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.067684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.067964 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] PATH_INFO: `/v3/projects/b34e45c472eb4f2b9e967053d0ff9a4a` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.069174 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34e45c472eb4f2b9e967053d0ff9a4a)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:50.071094 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:50.100896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:50.118625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34e45c472eb4f2b9e967053d0ff9a4a', 'request_id': 'req-ce7ce755-41fa-457b-adc1-47f7b5254a0a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:50.124188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34e45c472eb4f2b9e967053d0ff9a4a', 'request_id': 'req-ce7ce755-41fa-457b-adc1-47f7b5254a0a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:50.130252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34e45c472eb4f2b9e967053d0ff9a4a', 'request_id': 'req-ce7ce755-41fa-457b-adc1-47f7b5254a0a'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:50.143188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] The token cache is being invalidate because project b34e45c472eb4f2b9e967053d0ff9a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:17:50.143475 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:50.144117 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-ce7ce755-41fa-457b-adc1-47f7b5254a0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:50.145659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1387/2766] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:17:49 2022] DELETE /identity/v3/projects/b34e45c472eb4f2b9e967053d0ff9a4a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:50.151129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000874e-0dc3-4112-9127-0ef8379d48ae None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.151779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000874e-0dc3-4112-9127-0ef8379d48ae None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.152064 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000874e-0dc3-4112-9127-0ef8379d48ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.171941 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-0000874e-0dc3-4112-9127-0ef8379d48ae None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:50.173121 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0000874e-0dc3-4112-9127-0ef8379d48ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:50.231863 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1380/2767] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:50.239628 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca8425-ecc1-4057-a1cb-1c382b52fdfd None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.239918 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca8425-ecc1-4057-a1cb-1c382b52fdfd None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.240198 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca8425-ecc1-4057-a1cb-1c382b52fdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.258274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-faca8425-ecc1-4057-a1cb-1c382b52fdfd None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:50.259458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-faca8425-ecc1-4057-a1cb-1c382b52fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:50.268717 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1388/2768] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:17:50.272858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000874e-0dc3-4112-9127-0ef8379d48ae None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:50.283434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000874e-0dc3-4112-9127-0ef8379d48ae None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.287984 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fdf89-1428-42ba-b3d1-69287f302ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.288922 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdf89-1428-42ba-b3d1-69287f302ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.289212 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdf89-1428-42ba-b3d1-69287f302ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.289489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdf89-1428-42ba-b3d1-69287f302ac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.290245 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fdf89-1428-42ba-b3d1-69287f302ac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:17:50.290551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fdf89-1428-42ba-b3d1-69287f302ac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:50.292335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fdf89-1428-42ba-b3d1-69287f302ac2 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:50.303801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1381/2769] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:17:50.308099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca8425-ecc1-4057-a1cb-1c382b52fdfd None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:50.317306 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca8425-ecc1-4057-a1cb-1c382b52fdfd None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.321863 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3377a-d2e0-48b8-a7e3-2a892d53ec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.322987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3377a-d2e0-48b8-a7e3-2a892d53ec6d admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.323269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3377a-d2e0-48b8-a7e3-2a892d53ec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.323625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3377a-d2e0-48b8-a7e3-2a892d53ec6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.324352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3377a-d2e0-48b8-a7e3-2a892d53ec6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:50.326262 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3377a-d2e0-48b8-a7e3-2a892d53ec6d admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:50.350932 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1389/2770] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:17:50 2022] POST /identity/v3/projects => generated 363 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:50.355105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fdf89-1428-42ba-b3d1-69287f302ac2 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:50.365022 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fdf89-1428-42ba-b3d1-69287f302ac2 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.376877 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:50.421818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60b2ba-81aa-48a7-92a1-e29c0b40e711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.422555 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60b2ba-81aa-48a7-92a1-e29c0b40e711 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.422761 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60b2ba-81aa-48a7-92a1-e29c0b40e711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.423035 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60b2ba-81aa-48a7-92a1-e29c0b40e711 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.423714 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60b2ba-81aa-48a7-92a1-e29c0b40e711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:50.425644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60b2ba-81aa-48a7-92a1-e29c0b40e711 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:50.445696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1382/2771] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:17:50 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:17:50.450402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3377a-d2e0-48b8-a7e3-2a892d53ec6d admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:50.461091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3377a-d2e0-48b8-a7e3-2a892d53ec6d admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.465953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73adbb-42a4-4775-a3ca-5f572c200b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.466733 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73adbb-42a4-4775-a3ca-5f572c200b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.467027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73adbb-42a4-4775-a3ca-5f572c200b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.467308 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73adbb-42a4-4775-a3ca-5f572c200b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.468041 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73adbb-42a4-4775-a3ca-5f572c200b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:50.469677 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73adbb-42a4-4775-a3ca-5f572c200b81 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:50.480214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1390/2772] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:50.484023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60b2ba-81aa-48a7-92a1-e29c0b40e711 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:50.493853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60b2ba-81aa-48a7-92a1-e29c0b40e711 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.499047 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33750c41-9758-4f91-bda2-2adf4ce19f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.499702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33750c41-9758-4f91-bda2-2adf4ce19f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.499985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33750c41-9758-4f91-bda2-2adf4ce19f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.500261 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33750c41-9758-4f91-bda2-2adf4ce19f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.501021 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33750c41-9758-4f91-bda2-2adf4ce19f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:50.502647 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33750c41-9758-4f91-bda2-2adf4ce19f65 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:50.513189 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1383/2773] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:50.517357 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73adbb-42a4-4775-a3ca-5f572c200b81 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:50.527406 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73adbb-42a4-4775-a3ca-5f572c200b81 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.532060 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee73d3-c378-4ef5-978c-be4ef3a410c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.532972 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee73d3-c378-4ef5-978c-be4ef3a410c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.533261 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee73d3-c378-4ef5-978c-be4ef3a410c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.533538 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee73d3-c378-4ef5-978c-be4ef3a410c2 admin admin] PATH_INFO: `/v3/projects/32729d1171bb48a5b9d5cc3d0aed043b/users/da742f40460b4dd8ab9cab03b31760eb/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.548169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee73d3-c378-4ef5-978c-be4ef3a410c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32729d1171bb48a5b9d5cc3d0aed043b, user_id=da742f40460b4dd8ab9cab03b31760eb, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:50.550837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee73d3-c378-4ef5-978c-be4ef3a410c2 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:50.564215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1391/2774] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:17:50 2022] PUT /identity/v3/projects/32729d1171bb48a5b9d5cc3d0aed043b/users/da742f40460b4dd8ab9cab03b31760eb/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:50.624237 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53289d8e-37ba-4810-9233-e2b248e1a345 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.624563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53289d8e-37ba-4810-9233-e2b248e1a345 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.624845 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53289d8e-37ba-4810-9233-e2b248e1a345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.651008 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-53289d8e-37ba-4810-9233-e2b248e1a345 None None] MFA Rules not processed for user `da742f40460b4dd8ab9cab03b31760eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:50.652209 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-53289d8e-37ba-4810-9233-e2b248e1a345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:50.715724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1384/2775] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:50.720053 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee73d3-c378-4ef5-978c-be4ef3a410c2 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:50.789996 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee73d3-c378-4ef5-978c-be4ef3a410c2 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.795831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdec88-ec24-4271-a363-9a5f97a93295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.796508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdec88-ec24-4271-a363-9a5f97a93295 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.796803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdec88-ec24-4271-a363-9a5f97a93295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.797084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdec88-ec24-4271-a363-9a5f97a93295 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.798018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdec88-ec24-4271-a363-9a5f97a93295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:50.800367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdec88-ec24-4271-a363-9a5f97a93295 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:50.825684 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1392/2776] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:17:50 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:50.829652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53289d8e-37ba-4810-9233-e2b248e1a345 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:50.840918 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53289d8e-37ba-4810-9233-e2b248e1a345 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.902574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638df253-c427-4cad-acc3-02e54d27f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.903630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638df253-c427-4cad-acc3-02e54d27f439 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.903917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638df253-c427-4cad-acc3-02e54d27f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.904194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638df253-c427-4cad-acc3-02e54d27f439 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.904964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638df253-c427-4cad-acc3-02e54d27f439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:50.907324 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638df253-c427-4cad-acc3-02e54d27f439 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:50.928812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1385/2777] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:17:50 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:50.933066 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdec88-ec24-4271-a363-9a5f97a93295 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:50.943291 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdec88-ec24-4271-a363-9a5f97a93295 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.948546 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9da272-5755-4c42-9294-cc2acce5e3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.949203 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9da272-5755-4c42-9294-cc2acce5e3ec admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.949490 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9da272-5755-4c42-9294-cc2acce5e3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.949766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9da272-5755-4c42-9294-cc2acce5e3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.950488 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9da272-5755-4c42-9294-cc2acce5e3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:50.952096 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9da272-5755-4c42-9294-cc2acce5e3ec admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:50.964227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1393/2778] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:50.968425 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638df253-c427-4cad-acc3-02e54d27f439 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:50.978166 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638df253-c427-4cad-acc3-02e54d27f439 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:50.982823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d0106-2940-4ccb-8685-964d0a73bd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:50.983693 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d0106-2940-4ccb-8685-964d0a73bd89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:50.983971 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d0106-2940-4ccb-8685-964d0a73bd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:50.984241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d0106-2940-4ccb-8685-964d0a73bd89 admin admin] PATH_INFO: `/v3/projects/0f004f20477d495598d1b360893f2c44/users/69d20344fe544b9993df0dc32349b5b2/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:50.997077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:51.000566 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d0106-2940-4ccb-8685-964d0a73bd89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f004f20477d495598d1b360893f2c44, user_id=69d20344fe544b9993df0dc32349b5b2, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:51.003984 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d0106-2940-4ccb-8685-964d0a73bd89 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:51.017564 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1386/2779] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:17:50 2022] PUT /identity/v3/projects/0f004f20477d495598d1b360893f2c44/users/69d20344fe544b9993df0dc32349b5b2/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:51.021352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9da272-5755-4c42-9294-cc2acce5e3ec admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:51.031752 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9da272-5755-4c42-9294-cc2acce5e3ec admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:51.051517 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:51.090212 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6de71a-2ea4-4afa-afc6-f958d79e7e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:51.090883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6de71a-2ea4-4afa-afc6-f958d79e7e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:51.091144 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6de71a-2ea4-4afa-afc6-f958d79e7e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:51.091422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6de71a-2ea4-4afa-afc6-f958d79e7e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:51.092141 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6de71a-2ea4-4afa-afc6-f958d79e7e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:51.093793 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6de71a-2ea4-4afa-afc6-f958d79e7e49 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:51.103360 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1394/2780] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:51.108066 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d0106-2940-4ccb-8685-964d0a73bd89 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:51.118175 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d0106-2940-4ccb-8685-964d0a73bd89 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:51.123445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9153-5000-4f3a-99b2-763e8e2c37d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:51.124320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9153-5000-4f3a-99b2-763e8e2c37d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:51.124632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9153-5000-4f3a-99b2-763e8e2c37d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:51.124911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b9153-5000-4f3a-99b2-763e8e2c37d6 admin admin] PATH_INFO: `/v3/projects/0f004f20477d495598d1b360893f2c44/users/69d20344fe544b9993df0dc32349b5b2/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:51.126852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b9153-5000-4f3a-99b2-763e8e2c37d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f004f20477d495598d1b360893f2c44, user_id=69d20344fe544b9993df0dc32349b5b2, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:51.129508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b9153-5000-4f3a-99b2-763e8e2c37d6 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:51.139575 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1387/2781] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:17:51 2022] PUT /identity/v3/projects/0f004f20477d495598d1b360893f2c44/users/69d20344fe544b9993df0dc32349b5b2/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:51.149255 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408ad13-3deb-43ac-aeda-d5d2fceed9ab None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:51.149552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408ad13-3deb-43ac-aeda-d5d2fceed9ab None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:51.149829 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408ad13-3deb-43ac-aeda-d5d2fceed9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:51.179414 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-f408ad13-3deb-43ac-aeda-d5d2fceed9ab None None] MFA Rules not processed for user `69d20344fe544b9993df0dc32349b5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:51.180676 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-f408ad13-3deb-43ac-aeda-d5d2fceed9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:51.254840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1395/2782] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:51 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:51.265986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9153-5000-4f3a-99b2-763e8e2c37d6 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:51.267675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-685b9153-5000-4f3a-99b2-763e8e2c37d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:51.326434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b9153-5000-4f3a-99b2-763e8e2c37d6 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:51.330126 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e189811-77e0-4d2d-a9c3-80bea2fc7043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:51.330797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e189811-77e0-4d2d-a9c3-80bea2fc7043 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:51.331083 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e189811-77e0-4d2d-a9c3-80bea2fc7043 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:51.331360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e189811-77e0-4d2d-a9c3-80bea2fc7043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:51.341188 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e189811-77e0-4d2d-a9c3-80bea2fc7043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:51.342629 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e189811-77e0-4d2d-a9c3-80bea2fc7043 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:51.351898 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1388/2783] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:17:51 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:17:51.423236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408ad13-3deb-43ac-aeda-d5d2fceed9ab None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:51.433156 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408ad13-3deb-43ac-aeda-d5d2fceed9ab None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:51.436784 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5857736-9db4-4e79-ba81-6b5c6e86e5b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:51.437421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5857736-9db4-4e79-ba81-6b5c6e86e5b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:51.437708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5857736-9db4-4e79-ba81-6b5c6e86e5b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:51.437987 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5857736-9db4-4e79-ba81-6b5c6e86e5b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:51.500450 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5857736-9db4-4e79-ba81-6b5c6e86e5b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:51.501930 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5857736-9db4-4e79-ba81-6b5c6e86e5b0 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:51.511091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1396/2784] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:17:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:17:54.208025 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e189811-77e0-4d2d-a9c3-80bea2fc7043 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:54.271125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e189811-77e0-4d2d-a9c3-80bea2fc7043 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:54.275690 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:54.276589 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:54.276875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:54.277150 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] PATH_INFO: `/v3/users/da742f40460b4dd8ab9cab03b31760eb` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:54.278967 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] RBAC: Authorizing `identity:delete_user(user_id=da742f40460b4dd8ab9cab03b31760eb)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:54.280932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:54.327852 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da742f40460b4dd8ab9cab03b31760eb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:54.328525 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da742f40460b4dd8ab9cab03b31760eb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:54.335273 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da742f40460b4dd8ab9cab03b31760eb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:54.335868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da742f40460b4dd8ab9cab03b31760eb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:54.343042 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da742f40460b4dd8ab9cab03b31760eb'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:54.356553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:54.361336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1389/2785] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:17:54 2022] DELETE /identity/v3/users/da742f40460b4dd8ab9cab03b31760eb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:17:54.366073 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13002e8-ea9a-4c1b-a85e-ae7e7cccadcc None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:54.366604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13002e8-ea9a-4c1b-a85e-ae7e7cccadcc None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:54.366727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13002e8-ea9a-4c1b-a85e-ae7e7cccadcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:54.384209 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-e13002e8-ea9a-4c1b-a85e-ae7e7cccadcc None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:54.385432 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e13002e8-ea9a-4c1b-a85e-ae7e7cccadcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:54.426294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=100 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:54.445691 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1397/2786] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:54.455788 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:54.457267 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:54.513659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ab53f-98c3-4c9d-a82b-4bfa5c34f58c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:54.517284 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15b64a-f8d9-47fb-91cb-4273869c56da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:54.517919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15b64a-f8d9-47fb-91cb-4273869c56da service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:54.518198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15b64a-f8d9-47fb-91cb-4273869c56da service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:54.518470 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15b64a-f8d9-47fb-91cb-4273869c56da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:54.527675 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15b64a-f8d9-47fb-91cb-4273869c56da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:54.529283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15b64a-f8d9-47fb-91cb-4273869c56da service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:54.539146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1390/2787] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:17:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:17:54.803548 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13002e8-ea9a-4c1b-a85e-ae7e7cccadcc None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:54.805031 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-e13002e8-ea9a-4c1b-a85e-ae7e7cccadcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:54.817619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13002e8-ea9a-4c1b-a85e-ae7e7cccadcc None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:54.822256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:54.823150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:54.823436 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:54.823732 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] PATH_INFO: `/v3/projects/32729d1171bb48a5b9d5cc3d0aed043b` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:54.824998 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32729d1171bb48a5b9d5cc3d0aed043b)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:54.826953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:54.874640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32729d1171bb48a5b9d5cc3d0aed043b', 'request_id': 'req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:54.880150 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32729d1171bb48a5b9d5cc3d0aed043b', 'request_id': 'req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:54.886292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32729d1171bb48a5b9d5cc3d0aed043b', 'request_id': 'req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:54.898570 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] The token cache is being invalidate because project 32729d1171bb48a5b9d5cc3d0aed043b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:17:54.898862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:54.899489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:54.901061 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1398/2788] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:17:54 2022] DELETE /identity/v3/projects/32729d1171bb48a5b9d5cc3d0aed043b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:17:54.905679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15b64a-f8d9-47fb-91cb-4273869c56da service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:54.916652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15b64a-f8d9-47fb-91cb-4273869c56da service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:54.974226 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:54.975104 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:54.975389 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:54.975667 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] PATH_INFO: `/v3/users/69d20344fe544b9993df0dc32349b5b2` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:54.976960 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=69d20344fe544b9993df0dc32349b5b2)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:54.978870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:55.025379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69d20344fe544b9993df0dc32349b5b2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:55.026023 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69d20344fe544b9993df0dc32349b5b2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:55.032977 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69d20344fe544b9993df0dc32349b5b2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:55.033595 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69d20344fe544b9993df0dc32349b5b2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:55.040892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7c1baf56-2731-47a1-9049-bbe9b6ab8f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69d20344fe544b9993df0dc32349b5b2'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:55.058058 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1391/2789] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:17:54 2022] DELETE /identity/v3/users/69d20344fe544b9993df0dc32349b5b2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:55.324847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:55.326241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:55.338902 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f21d53-5c4b-4b78-91fe-ca640f5fc6e7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:55.395883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:55.396766 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:55.397049 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:55.397320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] PATH_INFO: `/v3/projects/0f004f20477d495598d1b360893f2c44` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:55.398480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f004f20477d495598d1b360893f2c44)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:55.400384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:55.448362 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f004f20477d495598d1b360893f2c44', 'request_id': 'req-9773500e-19bd-4525-be55-98d97e5469ff'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:55.454328 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f004f20477d495598d1b360893f2c44', 'request_id': 'req-9773500e-19bd-4525-be55-98d97e5469ff'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:55.460513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f004f20477d495598d1b360893f2c44', 'request_id': 'req-9773500e-19bd-4525-be55-98d97e5469ff'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:55.469674 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:55.474924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] The token cache is being invalidate because project 0f004f20477d495598d1b360893f2c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:17:55.475278 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:55.476185 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-9773500e-19bd-4525-be55-98d97e5469ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:17:55.478673 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1399/2790] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:17:55 2022] DELETE /identity/v3/projects/0f004f20477d495598d1b360893f2c44 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:55.483856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2aae9-fc81-43ff-8569-861b3ddee777 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:55.484335 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2aae9-fc81-43ff-8569-861b3ddee777 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:55.484474 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2aae9-fc81-43ff-8569-861b3ddee777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:55.502534 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-12f2aae9-fc81-43ff-8569-861b3ddee777 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:55.503672 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-12f2aae9-fc81-43ff-8569-861b3ddee777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:55.520350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:55.564108 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1392/2791] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:55.571922 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1209e5-ab0e-4acb-b656-eeddfb1465af None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:55.572215 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1209e5-ab0e-4acb-b656-eeddfb1465af None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:55.572552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1209e5-ab0e-4acb-b656-eeddfb1465af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:55.590367 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-6c1209e5-ab0e-4acb-b656-eeddfb1465af None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:55.591541 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6c1209e5-ab0e-4acb-b656-eeddfb1465af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:55.599320 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1400/2792] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:55.603360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2aae9-fc81-43ff-8569-861b3ddee777 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:55.613823 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2aae9-fc81-43ff-8569-861b3ddee777 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:55.618281 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d2a94-7393-44ec-9bcb-e67e027915fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:55.618928 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d2a94-7393-44ec-9bcb-e67e027915fe admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:55.619211 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d2a94-7393-44ec-9bcb-e67e027915fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:55.619484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d2a94-7393-44ec-9bcb-e67e027915fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:55.620197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d2a94-7393-44ec-9bcb-e67e027915fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:17:55.620537 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d2a94-7393-44ec-9bcb-e67e027915fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:55.623133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d2a94-7393-44ec-9bcb-e67e027915fe admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:55.634678 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1393/2793] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:17:55.638981 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1209e5-ab0e-4acb-b656-eeddfb1465af None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:55.648313 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1209e5-ab0e-4acb-b656-eeddfb1465af None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:55.653637 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d840f6-42bd-4a0b-ae61-20b58d5d6337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:55.654493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d840f6-42bd-4a0b-ae61-20b58d5d6337 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:55.654777 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d840f6-42bd-4a0b-ae61-20b58d5d6337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:55.655056 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d840f6-42bd-4a0b-ae61-20b58d5d6337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:55.655753 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d840f6-42bd-4a0b-ae61-20b58d5d6337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:55.658369 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d840f6-42bd-4a0b-ae61-20b58d5d6337 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:55.684412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1401/2794] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:17:55 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:55.688635 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d2a94-7393-44ec-9bcb-e67e027915fe admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:55.698475 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d2a94-7393-44ec-9bcb-e67e027915fe admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:55.753673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d011042-ebc6-4170-b132-81af57378655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:55.754265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d011042-ebc6-4170-b132-81af57378655 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:55.754546 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d011042-ebc6-4170-b132-81af57378655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:55.754822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d011042-ebc6-4170-b132-81af57378655 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:55.755500 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d011042-ebc6-4170-b132-81af57378655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:55.757439 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d011042-ebc6-4170-b132-81af57378655 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:55.776424 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1394/2795] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:17:55 2022] POST /identity/v3/users => generated 289 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:55.780982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d840f6-42bd-4a0b-ae61-20b58d5d6337 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:55.791329 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d840f6-42bd-4a0b-ae61-20b58d5d6337 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:55.796646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51877820-9528-4a17-929e-29b0d58a2213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:55.797381 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51877820-9528-4a17-929e-29b0d58a2213 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:55.797606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51877820-9528-4a17-929e-29b0d58a2213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:55.798068 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51877820-9528-4a17-929e-29b0d58a2213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:55.798852 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51877820-9528-4a17-929e-29b0d58a2213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:55.800967 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51877820-9528-4a17-929e-29b0d58a2213 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:55.811339 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1402/2796] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:55.815220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d011042-ebc6-4170-b132-81af57378655 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:55.825177 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d011042-ebc6-4170-b132-81af57378655 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:55.829773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d52c70-3135-40e0-9866-4a2678d5e189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:55.830423 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d52c70-3135-40e0-9866-4a2678d5e189 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:55.830707 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d52c70-3135-40e0-9866-4a2678d5e189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:55.830985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d52c70-3135-40e0-9866-4a2678d5e189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:55.831732 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d52c70-3135-40e0-9866-4a2678d5e189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:55.833346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d52c70-3135-40e0-9866-4a2678d5e189 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:55.843649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1395/2797] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:55.847511 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51877820-9528-4a17-929e-29b0d58a2213 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:55.858353 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51877820-9528-4a17-929e-29b0d58a2213 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:55.862910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0ddb3-c5e8-41a4-9a1d-3043acaa8d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:55.863800 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0ddb3-c5e8-41a4-9a1d-3043acaa8d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:55.864116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0ddb3-c5e8-41a4-9a1d-3043acaa8d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:55.864415 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0ddb3-c5e8-41a4-9a1d-3043acaa8d89 admin admin] PATH_INFO: `/v3/projects/8711a38c5c414b919a25c5554e3f3dc4/users/6903108fafaf4628a3f6f6f19234fa10/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:55.879034 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0ddb3-c5e8-41a4-9a1d-3043acaa8d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8711a38c5c414b919a25c5554e3f3dc4, user_id=6903108fafaf4628a3f6f6f19234fa10, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:55.881740 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0ddb3-c5e8-41a4-9a1d-3043acaa8d89 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:55.894551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1403/2798] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:17:55 2022] PUT /identity/v3/projects/8711a38c5c414b919a25c5554e3f3dc4/users/6903108fafaf4628a3f6f6f19234fa10/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:55.906192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4f90a-0134-4c6d-a351-084d65b7c923 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:55.906492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4f90a-0134-4c6d-a351-084d65b7c923 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:55.906777 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4f90a-0134-4c6d-a351-084d65b7c923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:55.934151 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-5ef4f90a-0134-4c6d-a351-084d65b7c923 None None] MFA Rules not processed for user `6903108fafaf4628a3f6f6f19234fa10`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:55.935350 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5ef4f90a-0134-4c6d-a351-084d65b7c923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:55.997989 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1396/2799] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:56.002441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0ddb3-c5e8-41a4-9a1d-3043acaa8d89 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.064597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0ddb3-c5e8-41a4-9a1d-3043acaa8d89 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.069137 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f847c5-5709-4488-ad7f-78de480468c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.069794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f847c5-5709-4488-ad7f-78de480468c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.070080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f847c5-5709-4488-ad7f-78de480468c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.070355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f847c5-5709-4488-ad7f-78de480468c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.071032 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f847c5-5709-4488-ad7f-78de480468c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.072969 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f847c5-5709-4488-ad7f-78de480468c9 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.098597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1404/2800] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:17:56 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:56.102624 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4f90a-0134-4c6d-a351-084d65b7c923 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.113645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4f90a-0134-4c6d-a351-084d65b7c923 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.126306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:56.171445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e797-3641-4fb2-a5e4-b9202d6f2238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.172114 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e797-3641-4fb2-a5e4-b9202d6f2238 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.172417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e797-3641-4fb2-a5e4-b9202d6f2238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.172691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e797-3641-4fb2-a5e4-b9202d6f2238 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.173369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e797-3641-4fb2-a5e4-b9202d6f2238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.175270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e797-3641-4fb2-a5e4-b9202d6f2238 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.194689 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1397/2801] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:17:56 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:56.198796 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f847c5-5709-4488-ad7f-78de480468c9 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.209792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f847c5-5709-4488-ad7f-78de480468c9 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.214579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f3619-7930-4e71-8fb0-ec5d36b983ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.215841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f3619-7930-4e71-8fb0-ec5d36b983ff admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.216226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f3619-7930-4e71-8fb0-ec5d36b983ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.216647 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f3619-7930-4e71-8fb0-ec5d36b983ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.217385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f3619-7930-4e71-8fb0-ec5d36b983ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.219025 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f3619-7930-4e71-8fb0-ec5d36b983ff admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.229643 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1405/2802] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:56.233703 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e797-3641-4fb2-a5e4-b9202d6f2238 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.243618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e797-3641-4fb2-a5e4-b9202d6f2238 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.249067 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713b025-d51d-46ff-9c01-c022126c85a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.250340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713b025-d51d-46ff-9c01-c022126c85a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.250627 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713b025-d51d-46ff-9c01-c022126c85a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.250904 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713b025-d51d-46ff-9c01-c022126c85a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.251633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713b025-d51d-46ff-9c01-c022126c85a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.253272 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713b025-d51d-46ff-9c01-c022126c85a1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.264495 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1398/2803] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:56.268627 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f3619-7930-4e71-8fb0-ec5d36b983ff admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.279324 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f3619-7930-4e71-8fb0-ec5d36b983ff admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.283965 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a960808-33e1-4644-895f-89d366bc2fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.284879 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a960808-33e1-4644-895f-89d366bc2fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.285189 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a960808-33e1-4644-895f-89d366bc2fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.285468 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a960808-33e1-4644-895f-89d366bc2fb4 admin admin] PATH_INFO: `/v3/projects/b913075a524f4160aa3c89b580398164/users/502df082572c40ab888ec0b43d265e90/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.300294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a960808-33e1-4644-895f-89d366bc2fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b913075a524f4160aa3c89b580398164, user_id=502df082572c40ab888ec0b43d265e90, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.303020 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a960808-33e1-4644-895f-89d366bc2fb4 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.316709 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1406/2804] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:17:56 2022] PUT /identity/v3/projects/b913075a524f4160aa3c89b580398164/users/502df082572c40ab888ec0b43d265e90/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:56.327437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38587e4b-6112-4c2d-8312-0d9630180d69 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.327735 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38587e4b-6112-4c2d-8312-0d9630180d69 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.328038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38587e4b-6112-4c2d-8312-0d9630180d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.356962 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-38587e4b-6112-4c2d-8312-0d9630180d69 None None] MFA Rules not processed for user `502df082572c40ab888ec0b43d265e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:56.358370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-38587e4b-6112-4c2d-8312-0d9630180d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:56.423452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1399/2805] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:56.428017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a960808-33e1-4644-895f-89d366bc2fb4 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.473434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:56.491227 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a960808-33e1-4644-895f-89d366bc2fb4 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.495792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7154c2-3329-4e90-8ad1-a0424903b5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.496521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7154c2-3329-4e90-8ad1-a0424903b5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.496839 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7154c2-3329-4e90-8ad1-a0424903b5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.497114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7154c2-3329-4e90-8ad1-a0424903b5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.497803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7154c2-3329-4e90-8ad1-a0424903b5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.499693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7154c2-3329-4e90-8ad1-a0424903b5fb admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.523970 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1407/2806] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:17:56 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:17:56.528512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38587e4b-6112-4c2d-8312-0d9630180d69 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.539178 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38587e4b-6112-4c2d-8312-0d9630180d69 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.594365 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ddc2-1ba5-4413-a5f5-507e7fc26f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.594966 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ddc2-1ba5-4413-a5f5-507e7fc26f1f admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.595252 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ddc2-1ba5-4413-a5f5-507e7fc26f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.595529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4ddc2-1ba5-4413-a5f5-507e7fc26f1f admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.596227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4ddc2-1ba5-4413-a5f5-507e7fc26f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.598204 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4ddc2-1ba5-4413-a5f5-507e7fc26f1f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.618915 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1400/2807] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:17:56 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:17:56.622298 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7154c2-3329-4e90-8ad1-a0424903b5fb admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.632461 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7154c2-3329-4e90-8ad1-a0424903b5fb admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.637050 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef2735-d6dc-4cf3-82bc-6c313fee49c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.637708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef2735-d6dc-4cf3-82bc-6c313fee49c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.637992 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef2735-d6dc-4cf3-82bc-6c313fee49c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.638270 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef2735-d6dc-4cf3-82bc-6c313fee49c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.638997 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef2735-d6dc-4cf3-82bc-6c313fee49c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.640619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef2735-d6dc-4cf3-82bc-6c313fee49c4 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.650920 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1408/2808] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:17:56.655531 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ddc2-1ba5-4413-a5f5-507e7fc26f1f admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.665541 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4ddc2-1ba5-4413-a5f5-507e7fc26f1f admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.670283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aa186-0534-4fcc-84fe-ac22e8b76eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.671153 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aa186-0534-4fcc-84fe-ac22e8b76eef admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.671435 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aa186-0534-4fcc-84fe-ac22e8b76eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.671711 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aa186-0534-4fcc-84fe-ac22e8b76eef admin admin] PATH_INFO: `/v3/projects/3e905bf5f04d4f0eab712894870b06f3/users/0b77d2baa5bb47839f1cea283f6af3a4/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.683668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:56.689417 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0aa186-0534-4fcc-84fe-ac22e8b76eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e905bf5f04d4f0eab712894870b06f3, user_id=0b77d2baa5bb47839f1cea283f6af3a4, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.693225 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0aa186-0534-4fcc-84fe-ac22e8b76eef admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.707029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1401/2809] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:17:56 2022] PUT /identity/v3/projects/3e905bf5f04d4f0eab712894870b06f3/users/0b77d2baa5bb47839f1cea283f6af3a4/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:56.710870 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef2735-d6dc-4cf3-82bc-6c313fee49c4 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.720941 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef2735-d6dc-4cf3-82bc-6c313fee49c4 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.776623 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2503377-b7c5-445a-b4c0-01319ef31563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.777297 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2503377-b7c5-445a-b4c0-01319ef31563 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.777653 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2503377-b7c5-445a-b4c0-01319ef31563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.777943 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2503377-b7c5-445a-b4c0-01319ef31563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.778672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2503377-b7c5-445a-b4c0-01319ef31563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.780274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2503377-b7c5-445a-b4c0-01319ef31563 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.790229 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1409/2810] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 08 02:17:56.794853 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aa186-0534-4fcc-84fe-ac22e8b76eef admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.806092 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aa186-0534-4fcc-84fe-ac22e8b76eef admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:56.811561 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821e1a-6749-4795-848a-8f780d34a02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:56.812605 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821e1a-6749-4795-848a-8f780d34a02b admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.812892 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821e1a-6749-4795-848a-8f780d34a02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.813170 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7821e1a-6749-4795-848a-8f780d34a02b admin admin] PATH_INFO: `/v3/projects/3e905bf5f04d4f0eab712894870b06f3/users/0b77d2baa5bb47839f1cea283f6af3a4/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.815218 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821e1a-6749-4795-848a-8f780d34a02b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e905bf5f04d4f0eab712894870b06f3, user_id=0b77d2baa5bb47839f1cea283f6af3a4, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:56.817935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7821e1a-6749-4795-848a-8f780d34a02b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:56.828344 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1402/2811] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:17:56 2022] PUT /identity/v3/projects/3e905bf5f04d4f0eab712894870b06f3/users/0b77d2baa5bb47839f1cea283f6af3a4/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:17:56.838248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de46fe-5884-40e4-80d5-e19913583020 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:56.838543 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de46fe-5884-40e4-80d5-e19913583020 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:56.838825 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de46fe-5884-40e4-80d5-e19913583020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:56.864924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-68de46fe-5884-40e4-80d5-e19913583020 None None] MFA Rules not processed for user `0b77d2baa5bb47839f1cea283f6af3a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:17:56.866059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-68de46fe-5884-40e4-80d5-e19913583020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:56.936387 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1410/2812] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:17:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:17:56.949563 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821e1a-6749-4795-848a-8f780d34a02b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:56.951917 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d7821e1a-6749-4795-848a-8f780d34a02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:57.009542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7821e1a-6749-4795-848a-8f780d34a02b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:57.013634 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9157-2743-467e-bcb4-66c13b8fe2e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:57.014269 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9157-2743-467e-bcb4-66c13b8fe2e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:57.014550 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9157-2743-467e-bcb4-66c13b8fe2e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:57.014822 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9157-2743-467e-bcb4-66c13b8fe2e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:57.023571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9157-2743-467e-bcb4-66c13b8fe2e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:57.025031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9157-2743-467e-bcb4-66c13b8fe2e8 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:57.034854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1403/2813] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:17:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:17:57.074510 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de46fe-5884-40e4-80d5-e19913583020 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:17:57.075957 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-68de46fe-5884-40e4-80d5-e19913583020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:17:57.087664 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de46fe-5884-40e4-80d5-e19913583020 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:17:57.091398 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1f504-a485-4250-8f6a-afe4bc23c133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:17:57.092037 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1f504-a485-4250-8f6a-afe4bc23c133 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:17:57.092315 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1f504-a485-4250-8f6a-afe4bc23c133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:17:57.092615 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1f504-a485-4250-8f6a-afe4bc23c133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:17:57.130817 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:17:57.149347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1f504-a485-4250-8f6a-afe4bc23c133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:17:57.150785 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1f504-a485-4250-8f6a-afe4bc23c133 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:17:57.160010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1411/2814] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:17:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:18:00.038366 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9157-2743-467e-bcb4-66c13b8fe2e8 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:00.099895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9157-2743-467e-bcb4-66c13b8fe2e8 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:00.104347 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:00.105358 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:00.105702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:00.106090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] PATH_INFO: `/v3/users/6903108fafaf4628a3f6f6f19234fa10` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:00.107265 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6903108fafaf4628a3f6f6f19234fa10)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:00.109227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:00.155331 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6903108fafaf4628a3f6f6f19234fa10'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.155973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6903108fafaf4628a3f6f6f19234fa10'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.162695 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6903108fafaf4628a3f6f6f19234fa10'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.163309 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6903108fafaf4628a3f6f6f19234fa10'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.171743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6903108fafaf4628a3f6f6f19234fa10'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.184832 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:00.189938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1404/2815] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:18:00 2022] DELETE /identity/v3/users/6903108fafaf4628a3f6f6f19234fa10 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:00.194638 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a099a5e-fb44-4413-9cdc-406d00c8c523 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:00.195219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a099a5e-fb44-4413-9cdc-406d00c8c523 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:00.195219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a099a5e-fb44-4413-9cdc-406d00c8c523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:00.212356 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-4a099a5e-fb44-4413-9cdc-406d00c8c523 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:00.213574 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4a099a5e-fb44-4413-9cdc-406d00c8c523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:00.272319 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1412/2816] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:00.282919 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:00.284317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:00.339942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa6ae-b983-455f-b416-7c840d52ebb8 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:00.344183 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f3595-4949-4f4c-bd95-9b68de77010a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:00.344856 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f3595-4949-4f4c-bd95-9b68de77010a service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:00.345137 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f3595-4949-4f4c-bd95-9b68de77010a service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:00.345410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f3595-4949-4f4c-bd95-9b68de77010a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:00.354253 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f3595-4949-4f4c-bd95-9b68de77010a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:00.356255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f3595-4949-4f4c-bd95-9b68de77010a service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:00.365491 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1405/2817] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:18:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:18:00.641300 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a099a5e-fb44-4413-9cdc-406d00c8c523 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:00.642903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4a099a5e-fb44-4413-9cdc-406d00c8c523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:00.655910 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a099a5e-fb44-4413-9cdc-406d00c8c523 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:00.660765 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:00.661656 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:00.661936 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:00.662213 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] PATH_INFO: `/v3/projects/8711a38c5c414b919a25c5554e3f3dc4` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:00.663402 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8711a38c5c414b919a25c5554e3f3dc4)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:00.665675 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:00.712891 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8711a38c5c414b919a25c5554e3f3dc4', 'request_id': 'req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.719346 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8711a38c5c414b919a25c5554e3f3dc4', 'request_id': 'req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.725618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8711a38c5c414b919a25c5554e3f3dc4', 'request_id': 'req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.738233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] The token cache is being invalidate because project 8711a38c5c414b919a25c5554e3f3dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:18:00.738523 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.739191 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.740745 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1413/2818] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:18:00 2022] DELETE /identity/v3/projects/8711a38c5c414b919a25c5554e3f3dc4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:00.744854 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f3595-4949-4f4c-bd95-9b68de77010a service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:00.754621 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f3595-4949-4f4c-bd95-9b68de77010a service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:00.812940 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:00.813805 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:00.814086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:00.814383 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] PATH_INFO: `/v3/users/502df082572c40ab888ec0b43d265e90` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:00.815549 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] RBAC: Authorizing `identity:delete_user(user_id=502df082572c40ab888ec0b43d265e90)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:00.817524 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:00.863886 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502df082572c40ab888ec0b43d265e90'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.864553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502df082572c40ab888ec0b43d265e90'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.872698 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502df082572c40ab888ec0b43d265e90'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.873336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502df082572c40ab888ec0b43d265e90'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.880748 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502df082572c40ab888ec0b43d265e90'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:00.898080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1406/2819] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:18:00 2022] DELETE /identity/v3/users/502df082572c40ab888ec0b43d265e90 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:01.174327 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:01.176033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:01.189489 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bcfdd-b32d-4323-a7ce-2f815479d8bb admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:01.250606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:01.251471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:01.251759 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:01.252039 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] PATH_INFO: `/v3/projects/b913075a524f4160aa3c89b580398164` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:01.253260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b913075a524f4160aa3c89b580398164)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:01.255188 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:01.257493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:01.305982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b913075a524f4160aa3c89b580398164', 'request_id': 'req-af900c8e-c4c2-496e-a2d0-c5920686986f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.312005 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b913075a524f4160aa3c89b580398164', 'request_id': 'req-af900c8e-c4c2-496e-a2d0-c5920686986f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.318220 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b913075a524f4160aa3c89b580398164', 'request_id': 'req-af900c8e-c4c2-496e-a2d0-c5920686986f'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.331694 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] The token cache is being invalidate because project b913075a524f4160aa3c89b580398164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:18:01.331978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.332629 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.334418 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1414/2820] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:18:01 2022] DELETE /identity/v3/projects/b913075a524f4160aa3c89b580398164 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:18:01.338120 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:01.339526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:01.352937 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a42aa1-e21e-401e-8a9b-02e319be204d admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:01.370685 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:01.412198 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:01.413101 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:01.413379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:01.413649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] PATH_INFO: `/v3/users/0b77d2baa5bb47839f1cea283f6af3a4` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:01.415070 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b77d2baa5bb47839f1cea283f6af3a4)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:01.417012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:01.461993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b77d2baa5bb47839f1cea283f6af3a4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.462629 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b77d2baa5bb47839f1cea283f6af3a4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.470167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b77d2baa5bb47839f1cea283f6af3a4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.470779 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b77d2baa5bb47839f1cea283f6af3a4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.477862 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-b6c9f5d3-202f-4266-8001-398154041f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b77d2baa5bb47839f1cea283f6af3a4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.495105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1407/2821] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:18:01 2022] DELETE /identity/v3/users/0b77d2baa5bb47839f1cea283f6af3a4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:01.757409 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:01.758862 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:01.773946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af900c8e-c4c2-496e-a2d0-c5920686986f admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:01.832120 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:01.833015 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:01.833303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:01.833582 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] PATH_INFO: `/v3/projects/3e905bf5f04d4f0eab712894870b06f3` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:01.834748 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e905bf5f04d4f0eab712894870b06f3)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:01.836692 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:01.881371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e905bf5f04d4f0eab712894870b06f3', 'request_id': 'req-5da335bc-ea92-4d1c-a42f-d45819c094aa'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.886810 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e905bf5f04d4f0eab712894870b06f3', 'request_id': 'req-5da335bc-ea92-4d1c-a42f-d45819c094aa'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.893893 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e905bf5f04d4f0eab712894870b06f3', 'request_id': 'req-5da335bc-ea92-4d1c-a42f-d45819c094aa'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.905595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] The token cache is being invalidate because project 3e905bf5f04d4f0eab712894870b06f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:18:01.905882 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.906586 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-5da335bc-ea92-4d1c-a42f-d45819c094aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:01.909080 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1415/2822] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:18:01 2022] DELETE /identity/v3/projects/3e905bf5f04d4f0eab712894870b06f3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:01.914437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308c8ab-50eb-444f-9f4b-68bcdb470579 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:01.914733 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308c8ab-50eb-444f-9f4b-68bcdb470579 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:01.915012 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308c8ab-50eb-444f-9f4b-68bcdb470579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:01.933255 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-8308c8ab-50eb-444f-9f4b-68bcdb470579 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:01.934941 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-8308c8ab-50eb-444f-9f4b-68bcdb470579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:01.992857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1408/2823] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:02.001777 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4fda-5e6d-40a0-82f6-e6e3048d72ca None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.002065 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4fda-5e6d-40a0-82f6-e6e3048d72ca None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.002344 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4fda-5e6d-40a0-82f6-e6e3048d72ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.020901 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-8f9f4fda-5e6d-40a0-82f6-e6e3048d72ca None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:02.022017 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8f9f4fda-5e6d-40a0-82f6-e6e3048d72ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:02.029804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1416/2824] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:02.033812 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308c8ab-50eb-444f-9f4b-68bcdb470579 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.044385 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308c8ab-50eb-444f-9f4b-68bcdb470579 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.048992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365986bb-e23b-4455-ac56-87577ef00465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.049651 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365986bb-e23b-4455-ac56-87577ef00465 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.049938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365986bb-e23b-4455-ac56-87577ef00465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.050241 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365986bb-e23b-4455-ac56-87577ef00465 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.050963 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365986bb-e23b-4455-ac56-87577ef00465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:18:02.051274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365986bb-e23b-4455-ac56-87577ef00465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.052911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365986bb-e23b-4455-ac56-87577ef00465 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.065338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1409/2825] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:18:02.069604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f4fda-5e6d-40a0-82f6-e6e3048d72ca None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.078779 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f4fda-5e6d-40a0-82f6-e6e3048d72ca None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.083421 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83dcc3-545a-4c41-b139-72209b7adae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.084059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83dcc3-545a-4c41-b139-72209b7adae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.084345 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83dcc3-545a-4c41-b139-72209b7adae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.084650 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83dcc3-545a-4c41-b139-72209b7adae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.085347 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83dcc3-545a-4c41-b139-72209b7adae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.087248 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83dcc3-545a-4c41-b139-72209b7adae7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.100092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:02.113380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1417/2826] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:18:02 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:18:02.117341 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365986bb-e23b-4455-ac56-87577ef00465 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.127767 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365986bb-e23b-4455-ac56-87577ef00465 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.153115 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:02.185773 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f26e2c-e3fc-4eae-a8ad-a87403fb1592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.186441 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f26e2c-e3fc-4eae-a8ad-a87403fb1592 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.186727 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f26e2c-e3fc-4eae-a8ad-a87403fb1592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.187000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f26e2c-e3fc-4eae-a8ad-a87403fb1592 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.187687 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f26e2c-e3fc-4eae-a8ad-a87403fb1592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.189630 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f26e2c-e3fc-4eae-a8ad-a87403fb1592 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.209414 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1410/2827] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:18:02 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:18:02.213401 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83dcc3-545a-4c41-b139-72209b7adae7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.223658 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83dcc3-545a-4c41-b139-72209b7adae7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.228216 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f9fd8-343c-48a0-8ff6-2b2b41323071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.228897 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f9fd8-343c-48a0-8ff6-2b2b41323071 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.229184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f9fd8-343c-48a0-8ff6-2b2b41323071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.229458 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f9fd8-343c-48a0-8ff6-2b2b41323071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.230179 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f9fd8-343c-48a0-8ff6-2b2b41323071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.232348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f9fd8-343c-48a0-8ff6-2b2b41323071 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.242569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1418/2828] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:18:02.246274 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f26e2c-e3fc-4eae-a8ad-a87403fb1592 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.257164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f26e2c-e3fc-4eae-a8ad-a87403fb1592 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.261745 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828807-6600-43c1-8351-dac7c5d68e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.262598 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828807-6600-43c1-8351-dac7c5d68e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.262878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828807-6600-43c1-8351-dac7c5d68e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.263151 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828807-6600-43c1-8351-dac7c5d68e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.263874 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d828807-6600-43c1-8351-dac7c5d68e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.265494 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d828807-6600-43c1-8351-dac7c5d68e37 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.275290 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1411/2829] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:18:02.279388 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f9fd8-343c-48a0-8ff6-2b2b41323071 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.290420 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f9fd8-343c-48a0-8ff6-2b2b41323071 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.295813 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82b329-cec5-4df8-9b93-792f8e591ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.296746 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82b329-cec5-4df8-9b93-792f8e591ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.297023 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82b329-cec5-4df8-9b93-792f8e591ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.297293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82b329-cec5-4df8-9b93-792f8e591ccb admin admin] PATH_INFO: `/v3/projects/fa04a23817b64781a1a803e625d28882/users/118d7e768c1d4170ba54d3457369aac9/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.312033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82b329-cec5-4df8-9b93-792f8e591ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa04a23817b64781a1a803e625d28882, user_id=118d7e768c1d4170ba54d3457369aac9, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.314794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82b329-cec5-4df8-9b93-792f8e591ccb admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.327350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1419/2830] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:18:02 2022] PUT /identity/v3/projects/fa04a23817b64781a1a803e625d28882/users/118d7e768c1d4170ba54d3457369aac9/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:02.338601 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7afe93-bd63-48b3-8035-09bb739b7742 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.339103 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7afe93-bd63-48b3-8035-09bb739b7742 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.339207 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7afe93-bd63-48b3-8035-09bb739b7742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.375542 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-eb7afe93-bd63-48b3-8035-09bb739b7742 None None] MFA Rules not processed for user `118d7e768c1d4170ba54d3457369aac9`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:02.376768 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-eb7afe93-bd63-48b3-8035-09bb739b7742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:02.440708 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1412/2831] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:02.445092 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82b329-cec5-4df8-9b93-792f8e591ccb admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.506842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82b329-cec5-4df8-9b93-792f8e591ccb admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.511544 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1f78f-6eec-4c70-93b6-2efbb0f1f75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.512249 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1f78f-6eec-4c70-93b6-2efbb0f1f75b admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.512566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1f78f-6eec-4c70-93b6-2efbb0f1f75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.512853 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1f78f-6eec-4c70-93b6-2efbb0f1f75b admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.513531 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1f78f-6eec-4c70-93b6-2efbb0f1f75b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.515425 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1f78f-6eec-4c70-93b6-2efbb0f1f75b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.540769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1420/2832] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:18:02 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:18:02.545399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7afe93-bd63-48b3-8035-09bb739b7742 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.555992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7afe93-bd63-48b3-8035-09bb739b7742 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.612277 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414a562-faf8-4004-b039-14b73f96af66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.612973 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414a562-faf8-4004-b039-14b73f96af66 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.613258 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414a562-faf8-4004-b039-14b73f96af66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.613536 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414a562-faf8-4004-b039-14b73f96af66 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.614219 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414a562-faf8-4004-b039-14b73f96af66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.616175 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414a562-faf8-4004-b039-14b73f96af66 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.635239 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1413/2833] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:18:02 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:18:02.639596 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1f78f-6eec-4c70-93b6-2efbb0f1f75b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.649707 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1f78f-6eec-4c70-93b6-2efbb0f1f75b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.654966 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee49514-805a-40f4-a6a3-8670a27beecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.655618 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee49514-805a-40f4-a6a3-8670a27beecf admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.655899 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee49514-805a-40f4-a6a3-8670a27beecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.656171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee49514-805a-40f4-a6a3-8670a27beecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.656928 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee49514-805a-40f4-a6a3-8670a27beecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.658513 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee49514-805a-40f4-a6a3-8670a27beecf admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.669723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1421/2834] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:18:02.673857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414a562-faf8-4004-b039-14b73f96af66 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.684499 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414a562-faf8-4004-b039-14b73f96af66 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.689053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95540c-7eb0-456a-9e81-fc74a0dd0d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.689935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95540c-7eb0-456a-9e81-fc74a0dd0d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.690220 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95540c-7eb0-456a-9e81-fc74a0dd0d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.690493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95540c-7eb0-456a-9e81-fc74a0dd0d37 admin admin] PATH_INFO: `/v3/projects/98d1248f792247c5abc614def00bb394/users/f48766379c34420595a5624dae9365fd/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.710345 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95540c-7eb0-456a-9e81-fc74a0dd0d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d1248f792247c5abc614def00bb394, user_id=f48766379c34420595a5624dae9365fd, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.713094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95540c-7eb0-456a-9e81-fc74a0dd0d37 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.728869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1414/2835] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:18:02 2022] PUT /identity/v3/projects/98d1248f792247c5abc614def00bb394/users/f48766379c34420595a5624dae9365fd/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:02.733670 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee49514-805a-40f4-a6a3-8670a27beecf admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.743873 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee49514-805a-40f4-a6a3-8670a27beecf admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.801238 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25675b3e-a10c-4333-930f-0aa4f5cd5ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.801831 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25675b3e-a10c-4333-930f-0aa4f5cd5ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.802116 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25675b3e-a10c-4333-930f-0aa4f5cd5ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.802391 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25675b3e-a10c-4333-930f-0aa4f5cd5ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.803119 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25675b3e-a10c-4333-930f-0aa4f5cd5ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.804757 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25675b3e-a10c-4333-930f-0aa4f5cd5ef4 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.814683 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1422/2836] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:18:02.818964 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95540c-7eb0-456a-9e81-fc74a0dd0d37 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.824961 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:02.831664 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95540c-7eb0-456a-9e81-fc74a0dd0d37 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:02.836724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f4b32-ef26-4e80-b500-68929bfafb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:02.837633 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f4b32-ef26-4e80-b500-68929bfafb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.837914 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f4b32-ef26-4e80-b500-68929bfafb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.838206 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f4b32-ef26-4e80-b500-68929bfafb86 admin admin] PATH_INFO: `/v3/projects/98d1248f792247c5abc614def00bb394/users/f48766379c34420595a5624dae9365fd/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.840756 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f4b32-ef26-4e80-b500-68929bfafb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d1248f792247c5abc614def00bb394, user_id=f48766379c34420595a5624dae9365fd, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:02.843420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f4b32-ef26-4e80-b500-68929bfafb86 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:02.853457 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1415/2837] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:18:02 2022] PUT /identity/v3/projects/98d1248f792247c5abc614def00bb394/users/f48766379c34420595a5624dae9365fd/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:02.864059 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8146922-bcaa-4237-82be-0ecfb59a8b16 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:02.864350 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8146922-bcaa-4237-82be-0ecfb59a8b16 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:02.864663 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8146922-bcaa-4237-82be-0ecfb59a8b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:02.891537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-a8146922-bcaa-4237-82be-0ecfb59a8b16 None None] MFA Rules not processed for user `f48766379c34420595a5624dae9365fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:02.892804 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a8146922-bcaa-4237-82be-0ecfb59a8b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:02.909924 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:02.963765 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1423/2838] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:02.975192 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f4b32-ef26-4e80-b500-68929bfafb86 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:02.976943 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-fb7f4b32-ef26-4e80-b500-68929bfafb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:03.038603 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f4b32-ef26-4e80-b500-68929bfafb86 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:03.042466 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ab873-1ec6-4f80-a8be-6b068e12514f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:03.043164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ab873-1ec6-4f80-a8be-6b068e12514f service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:03.043551 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ab873-1ec6-4f80-a8be-6b068e12514f service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:03.043836 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ab873-1ec6-4f80-a8be-6b068e12514f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:03.102578 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ab873-1ec6-4f80-a8be-6b068e12514f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:03.103948 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ab873-1ec6-4f80-a8be-6b068e12514f service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:03.112950 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1416/2839] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:18:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 08 02:18:07.262553 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8146922-bcaa-4237-82be-0ecfb59a8b16 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:07.265728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a8146922-bcaa-4237-82be-0ecfb59a8b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:07.281486 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8146922-bcaa-4237-82be-0ecfb59a8b16 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:07.285290 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6614ca7-63c5-4a5e-bd35-6e289ddf26e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:07.285937 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6614ca7-63c5-4a5e-bd35-6e289ddf26e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:07.286219 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6614ca7-63c5-4a5e-bd35-6e289ddf26e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:07.286579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6614ca7-63c5-4a5e-bd35-6e289ddf26e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:07.296803 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6614ca7-63c5-4a5e-bd35-6e289ddf26e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:07.298236 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6614ca7-63c5-4a5e-bd35-6e289ddf26e3 service cinder] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:07.307734 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1424/2840] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:18:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:18:12.160676 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ab873-1ec6-4f80-a8be-6b068e12514f service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:12.222801 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ab873-1ec6-4f80-a8be-6b068e12514f service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:12.227269 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:12.228129 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:12.228440 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:12.228702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] PATH_INFO: `/v3/users/118d7e768c1d4170ba54d3457369aac9` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:12.229869 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=118d7e768c1d4170ba54d3457369aac9)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:12.231810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:12.276000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:12.277618 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118d7e768c1d4170ba54d3457369aac9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.278452 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118d7e768c1d4170ba54d3457369aac9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.287236 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118d7e768c1d4170ba54d3457369aac9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.287863 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118d7e768c1d4170ba54d3457369aac9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.295099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118d7e768c1d4170ba54d3457369aac9'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.312739 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1417/2841] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:18:12 2022] DELETE /identity/v3/users/118d7e768c1d4170ba54d3457369aac9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:12.317321 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01e38-208e-41ce-b571-c1b1323a16db None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:12.317805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01e38-208e-41ce-b571-c1b1323a16db None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:12.317908 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01e38-208e-41ce-b571-c1b1323a16db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:12.335856 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-b1d01e38-208e-41ce-b571-c1b1323a16db None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:12.337086 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b1d01e38-208e-41ce-b571-c1b1323a16db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:12.396282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1425/2842] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:12.405641 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:12.407057 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:12.464783 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf63574-3161-4547-938f-9877540edc85 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:12.468582 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb6c08-c624-4dc6-85de-2c5b7cc981be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:12.469234 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb6c08-c624-4dc6-85de-2c5b7cc981be service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:12.469517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb6c08-c624-4dc6-85de-2c5b7cc981be service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:12.469794 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb6c08-c624-4dc6-85de-2c5b7cc981be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:12.478790 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb6c08-c624-4dc6-85de-2c5b7cc981be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:12.480213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb6c08-c624-4dc6-85de-2c5b7cc981be service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:12.490406 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1418/2843] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:18:12.759916 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d01e38-208e-41ce-b571-c1b1323a16db None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:12.761408 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-b1d01e38-208e-41ce-b571-c1b1323a16db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:12.776903 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d01e38-208e-41ce-b571-c1b1323a16db None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:12.783214 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:12.784267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:12.784613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:12.784844 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] PATH_INFO: `/v3/projects/fa04a23817b64781a1a803e625d28882` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:12.786063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa04a23817b64781a1a803e625d28882)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:12.788004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:12.835780 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa04a23817b64781a1a803e625d28882', 'request_id': 'req-d58fda60-17f6-4831-94f8-e64dbfc65547'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.841452 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa04a23817b64781a1a803e625d28882', 'request_id': 'req-d58fda60-17f6-4831-94f8-e64dbfc65547'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.847495 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa04a23817b64781a1a803e625d28882', 'request_id': 'req-d58fda60-17f6-4831-94f8-e64dbfc65547'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.859480 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] The token cache is being invalidate because project fa04a23817b64781a1a803e625d28882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:18:12.859767 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.860377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.861942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1426/2844] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:18:12 2022] DELETE /identity/v3/projects/fa04a23817b64781a1a803e625d28882 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:12.866030 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb6c08-c624-4dc6-85de-2c5b7cc981be service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:12.876125 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb6c08-c624-4dc6-85de-2c5b7cc981be service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:12.932696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:12.933650 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:12.933883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:12.934157 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] PATH_INFO: `/v3/users/f48766379c34420595a5624dae9365fd` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:12.936028 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48766379c34420595a5624dae9365fd)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:12.937983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:12.982706 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48766379c34420595a5624dae9365fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.983410 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48766379c34420595a5624dae9365fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.990199 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48766379c34420595a5624dae9365fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.990824 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48766379c34420595a5624dae9365fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:12.998487 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-cea11030-8067-4a2a-8f73-7ab7b5c5de26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48766379c34420595a5624dae9365fd'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:13.007632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:13.020513 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1419/2845] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:18:12 2022] DELETE /identity/v3/users/f48766379c34420595a5624dae9365fd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:13.289737 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:13.291166 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:13.304953 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fda60-17f6-4831-94f8-e64dbfc65547 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:13.326646 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:13.366254 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:13.367125 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:13.367407 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:13.367680 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] PATH_INFO: `/v3/projects/98d1248f792247c5abc614def00bb394` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:13.368883 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d1248f792247c5abc614def00bb394)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:13.370851 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:13.417775 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d1248f792247c5abc614def00bb394', 'request_id': 'req-16d70b7e-b832-46da-bdd6-f6f344feb6d6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:13.424033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d1248f792247c5abc614def00bb394', 'request_id': 'req-16d70b7e-b832-46da-bdd6-f6f344feb6d6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:13.430142 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d1248f792247c5abc614def00bb394', 'request_id': 'req-16d70b7e-b832-46da-bdd6-f6f344feb6d6'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:13.443040 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] The token cache is being invalidate because project 98d1248f792247c5abc614def00bb394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:18:13.443326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:13.443946 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-16d70b7e-b832-46da-bdd6-f6f344feb6d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:13.446033 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1427/2846] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:18:13 2022] DELETE /identity/v3/projects/98d1248f792247c5abc614def00bb394 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:13.452276 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107eb0b9-2bf8-4bc8-90e4-424da9ec2567 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:13.452795 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107eb0b9-2bf8-4bc8-90e4-424da9ec2567 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:13.452896 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107eb0b9-2bf8-4bc8-90e4-424da9ec2567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:13.469998 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-107eb0b9-2bf8-4bc8-90e4-424da9ec2567 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:13.471134 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-107eb0b9-2bf8-4bc8-90e4-424da9ec2567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:13.529018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1420/2847] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 08 02:18:13.537429 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2df2-e43a-4949-9df7-ae842bcea044 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:13.537723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2df2-e43a-4949-9df7-ae842bcea044 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:13.538004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2df2-e43a-4949-9df7-ae842bcea044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:13.556847 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-76dd2df2-e43a-4949-9df7-ae842bcea044 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:13.558045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-76dd2df2-e43a-4949-9df7-ae842bcea044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:13.566567 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1428/2848] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:13.570663 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107eb0b9-2bf8-4bc8-90e4-424da9ec2567 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:13.580294 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107eb0b9-2bf8-4bc8-90e4-424da9ec2567 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:13.584897 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574d2ef-fafd-4b66-9a79-3bb3aa2a5267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:13.585540 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574d2ef-fafd-4b66-9a79-3bb3aa2a5267 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:13.585818 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574d2ef-fafd-4b66-9a79-3bb3aa2a5267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:13.586087 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574d2ef-fafd-4b66-9a79-3bb3aa2a5267 admin admin] PATH_INFO: `/v3/domains` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:13.586792 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574d2ef-fafd-4b66-9a79-3bb3aa2a5267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:18:13.587091 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574d2ef-fafd-4b66-9a79-3bb3aa2a5267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:13.588697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574d2ef-fafd-4b66-9a79-3bb3aa2a5267 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:13.601086 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1421/2849] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:18:13.605551 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd2df2-e43a-4949-9df7-ae842bcea044 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:13.615422 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd2df2-e43a-4949-9df7-ae842bcea044 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:13.620348 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6209e5f-0189-448e-b863-5e447e8e9a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:13.621027 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209e5f-0189-448e-b863-5e447e8e9a33 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:13.621312 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209e5f-0189-448e-b863-5e447e8e9a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:13.621585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209e5f-0189-448e-b863-5e447e8e9a33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:13.622294 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6209e5f-0189-448e-b863-5e447e8e9a33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:13.624659 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6209e5f-0189-448e-b863-5e447e8e9a33 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:13.650292 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1429/2850] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:18:13 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:18:13.656708 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574d2ef-fafd-4b66-9a79-3bb3aa2a5267 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:13.667172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574d2ef-fafd-4b66-9a79-3bb3aa2a5267 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:13.723747 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80637d-3ab2-4699-a12e-db115e91dd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:13.724342 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80637d-3ab2-4699-a12e-db115e91dd34 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:13.724659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80637d-3ab2-4699-a12e-db115e91dd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:13.724938 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80637d-3ab2-4699-a12e-db115e91dd34 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:13.725638 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80637d-3ab2-4699-a12e-db115e91dd34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:13.727570 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80637d-3ab2-4699-a12e-db115e91dd34 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:13.747379 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1422/2851] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:18:13 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:18:13.751769 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6209e5f-0189-448e-b863-5e447e8e9a33 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:13.761986 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6209e5f-0189-448e-b863-5e447e8e9a33 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:13.766805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ebdd-7041-41ce-b1e6-19fb3021006c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:13.767471 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ebdd-7041-41ce-b1e6-19fb3021006c admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:13.767761 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ebdd-7041-41ce-b1e6-19fb3021006c admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:13.768044 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ebdd-7041-41ce-b1e6-19fb3021006c admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:13.768942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111ebdd-7041-41ce-b1e6-19fb3021006c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:13.770562 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111ebdd-7041-41ce-b1e6-19fb3021006c admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:13.781467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1430/2852] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:18:13.785288 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80637d-3ab2-4699-a12e-db115e91dd34 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:13.796517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80637d-3ab2-4699-a12e-db115e91dd34 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:13.801694 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14886d4-192e-4c8a-a3c2-4198bb9ae13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:13.802796 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14886d4-192e-4c8a-a3c2-4198bb9ae13c admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:13.803094 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14886d4-192e-4c8a-a3c2-4198bb9ae13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:13.803371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14886d4-192e-4c8a-a3c2-4198bb9ae13c admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:13.804099 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14886d4-192e-4c8a-a3c2-4198bb9ae13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:13.805752 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14886d4-192e-4c8a-a3c2-4198bb9ae13c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:13.816143 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1423/2853] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:18:13.820171 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ebdd-7041-41ce-b1e6-19fb3021006c admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:13.830364 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ebdd-7041-41ce-b1e6-19fb3021006c admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:13.834915 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb332e3c-9d82-4323-8588-ec44789c97f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:13.835815 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb332e3c-9d82-4323-8588-ec44789c97f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:13.836100 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb332e3c-9d82-4323-8588-ec44789c97f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:13.836376 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb332e3c-9d82-4323-8588-ec44789c97f7 admin admin] PATH_INFO: `/v3/projects/78fedd985d3b4bebbf0aefecc7b2f84b/users/653cc3b959494b05b34f2d4afb19861e/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:13.851478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb332e3c-9d82-4323-8588-ec44789c97f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78fedd985d3b4bebbf0aefecc7b2f84b, user_id=653cc3b959494b05b34f2d4afb19861e, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:13.854169 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb332e3c-9d82-4323-8588-ec44789c97f7 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:13.868794 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1431/2854] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:18:13 2022] PUT /identity/v3/projects/78fedd985d3b4bebbf0aefecc7b2f84b/users/653cc3b959494b05b34f2d4afb19861e/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:13.880334 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c95ef-77c7-426c-a683-73ec110d7162 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:13.880668 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c95ef-77c7-426c-a683-73ec110d7162 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:13.880954 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c95ef-77c7-426c-a683-73ec110d7162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:13.908371 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-bc3c95ef-77c7-426c-a683-73ec110d7162 None None] MFA Rules not processed for user `653cc3b959494b05b34f2d4afb19861e`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:13.909726 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-bc3c95ef-77c7-426c-a683-73ec110d7162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:13.953512 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:13.979680 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1424/2855] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:13.985035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb332e3c-9d82-4323-8588-ec44789c97f7 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:14.035426 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:14.048991 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb332e3c-9d82-4323-8588-ec44789c97f7 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:14.053597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c151c-c26d-4726-8d87-8d4d9c832f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:14.054252 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c151c-c26d-4726-8d87-8d4d9c832f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:14.054533 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c151c-c26d-4726-8d87-8d4d9c832f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:14.054805 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c151c-c26d-4726-8d87-8d4d9c832f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:14.055492 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c151c-c26d-4726-8d87-8d4d9c832f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:14.057512 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c151c-c26d-4726-8d87-8d4d9c832f3f admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:14.082889 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1432/2856] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:18:13 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:18:14.087133 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c95ef-77c7-426c-a683-73ec110d7162 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:14.098648 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c95ef-77c7-426c-a683-73ec110d7162 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:14.153018 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9732f-564e-4092-83be-9800370ff1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:14.154353 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9732f-564e-4092-83be-9800370ff1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:14.154722 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9732f-564e-4092-83be-9800370ff1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:14.155000 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9732f-564e-4092-83be-9800370ff1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:14.155681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9732f-564e-4092-83be-9800370ff1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:14.157645 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9732f-564e-4092-83be-9800370ff1a1 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:14.177389 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1425/2857] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:18:14 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:18:14.181334 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c151c-c26d-4726-8d87-8d4d9c832f3f admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:14.192556 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c151c-c26d-4726-8d87-8d4d9c832f3f admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:14.197208 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de6a9f-aa9a-4873-a964-610680d6ed48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:14.197874 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de6a9f-aa9a-4873-a964-610680d6ed48 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:14.198158 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de6a9f-aa9a-4873-a964-610680d6ed48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:14.198434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de6a9f-aa9a-4873-a964-610680d6ed48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:14.199183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de6a9f-aa9a-4873-a964-610680d6ed48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:14.201569 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de6a9f-aa9a-4873-a964-610680d6ed48 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:14.212084 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1433/2858] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:18:14.217368 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9732f-564e-4092-83be-9800370ff1a1 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:14.227184 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9732f-564e-4092-83be-9800370ff1a1 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:14.232015 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050584f-ad9c-4b4c-8cb9-2dd817c97f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:14.232881 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050584f-ad9c-4b4c-8cb9-2dd817c97f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:14.233160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050584f-ad9c-4b4c-8cb9-2dd817c97f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:14.233434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050584f-ad9c-4b4c-8cb9-2dd817c97f84 admin admin] PATH_INFO: `/v3/projects/d9c77ac564434ac8b4756d362436fab5/users/a6f10b01611842d0bda15a1c669b0154/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:14.249983 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050584f-ad9c-4b4c-8cb9-2dd817c97f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c77ac564434ac8b4756d362436fab5, user_id=a6f10b01611842d0bda15a1c669b0154, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:14.252728 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050584f-ad9c-4b4c-8cb9-2dd817c97f84 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:14.266479 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1426/2859] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:18:14 2022] PUT /identity/v3/projects/d9c77ac564434ac8b4756d362436fab5/users/a6f10b01611842d0bda15a1c669b0154/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:14.271530 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de6a9f-aa9a-4873-a964-610680d6ed48 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:14.286595 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de6a9f-aa9a-4873-a964-610680d6ed48 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:14.356175 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27c0e6-12cd-4135-8d3f-be102594ff9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:14.356975 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27c0e6-12cd-4135-8d3f-be102594ff9a admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:14.357232 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27c0e6-12cd-4135-8d3f-be102594ff9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:14.357508 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27c0e6-12cd-4135-8d3f-be102594ff9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:14.358228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27c0e6-12cd-4135-8d3f-be102594ff9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:14.359819 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27c0e6-12cd-4135-8d3f-be102594ff9a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:14.369457 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1434/2860] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:18:14.374071 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050584f-ad9c-4b4c-8cb9-2dd817c97f84 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:14.384719 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050584f-ad9c-4b4c-8cb9-2dd817c97f84 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:14.390228 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456be1de-2fd7-4e4d-9a64-956aeeeb571a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:14.391147 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be1de-2fd7-4e4d-9a64-956aeeeb571a admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:14.391427 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be1de-2fd7-4e4d-9a64-956aeeeb571a admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:14.391700 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be1de-2fd7-4e4d-9a64-956aeeeb571a admin admin] PATH_INFO: `/v3/projects/d9c77ac564434ac8b4756d362436fab5/users/a6f10b01611842d0bda15a1c669b0154/roles/71044d1835ec4f909f32dbffcfe31811` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:14.393712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456be1de-2fd7-4e4d-9a64-956aeeeb571a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c77ac564434ac8b4756d362436fab5, user_id=a6f10b01611842d0bda15a1c669b0154, role_id=71044d1835ec4f909f32dbffcfe31811)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:14.396367 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456be1de-2fd7-4e4d-9a64-956aeeeb571a admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:14.406932 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1427/2861] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:18:14 2022] PUT /identity/v3/projects/d9c77ac564434ac8b4756d362436fab5/users/a6f10b01611842d0bda15a1c669b0154/roles/71044d1835ec4f909f32dbffcfe31811 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:14.418052 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9597fa-af3c-4841-a2b0-3fd1b56fc6e4 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:14.418342 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9597fa-af3c-4841-a2b0-3fd1b56fc6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:14.418616 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9597fa-af3c-4841-a2b0-3fd1b56fc6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:14.444842 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-8a9597fa-af3c-4841-a2b0-3fd1b56fc6e4 None None] MFA Rules not processed for user `a6f10b01611842d0bda15a1c669b0154`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:14.446064 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-8a9597fa-af3c-4841-a2b0-3fd1b56fc6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:14.514218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1435/2862] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:14.525077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456be1de-2fd7-4e4d-9a64-956aeeeb571a admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:14.526797 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-456be1de-2fd7-4e4d-9a64-956aeeeb571a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:14.582352 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456be1de-2fd7-4e4d-9a64-956aeeeb571a admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:14.585985 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8ff70-7e4b-4c1a-8643-a4aeb0973ada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:14.586622 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8ff70-7e4b-4c1a-8643-a4aeb0973ada service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:14.586906 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8ff70-7e4b-4c1a-8643-a4aeb0973ada service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:14.587180 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8ff70-7e4b-4c1a-8643-a4aeb0973ada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:14.630754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:14.643233 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8ff70-7e4b-4c1a-8643-a4aeb0973ada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:14.644697 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8ff70-7e4b-4c1a-8643-a4aeb0973ada service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:14.653412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1428/2863] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:18:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:18:17.164885 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9597fa-af3c-4841-a2b0-3fd1b56fc6e4 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:17.212799 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:17.229043 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9597fa-af3c-4841-a2b0-3fd1b56fc6e4 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:17.233672 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:17.234537 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:17.234823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:17.235102 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] PATH_INFO: `/v3/users/653cc3b959494b05b34f2d4afb19861e` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:17.236293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] RBAC: Authorizing `identity:delete_user(user_id=653cc3b959494b05b34f2d4afb19861e)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:17.238265 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:17.283716 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '653cc3b959494b05b34f2d4afb19861e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:17.284371 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '653cc3b959494b05b34f2d4afb19861e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:17.291552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '653cc3b959494b05b34f2d4afb19861e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:17.292182 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '653cc3b959494b05b34f2d4afb19861e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:17.304473 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '653cc3b959494b05b34f2d4afb19861e'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:17.316906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1436/2864] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:18:17 2022] DELETE /identity/v3/users/653cc3b959494b05b34f2d4afb19861e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:17.321449 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02576415-4eea-4c15-b5ac-a92569562856 None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:17.321959 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02576415-4eea-4c15-b5ac-a92569562856 None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:17.322080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02576415-4eea-4c15-b5ac-a92569562856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:17.339172 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-02576415-4eea-4c15-b5ac-a92569562856 None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:17.340359 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-02576415-4eea-4c15-b5ac-a92569562856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:17.401430 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1429/2865] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:17.411385 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:17.412840 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:17.473755 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6190e-7843-4911-87d7-e0862b94610a admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:17.477374 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607e3e1-f5de-4794-9fcd-b413a806a237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:17.478021 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607e3e1-f5de-4794-9fcd-b413a806a237 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:17.478304 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607e3e1-f5de-4794-9fcd-b413a806a237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:17.478580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607e3e1-f5de-4794-9fcd-b413a806a237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:17.487536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607e3e1-f5de-4794-9fcd-b413a806a237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:17.489018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607e3e1-f5de-4794-9fcd-b413a806a237 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:17.498942 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1437/2866] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:18:17.772776 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02576415-4eea-4c15-b5ac-a92569562856 None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:17.774420 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-02576415-4eea-4c15-b5ac-a92569562856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:17.793567 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02576415-4eea-4c15-b5ac-a92569562856 None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:17.804317 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:17.806164 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:17.806517 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:17.806993 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] PATH_INFO: `/v3/projects/78fedd985d3b4bebbf0aefecc7b2f84b` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:17.808724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78fedd985d3b4bebbf0aefecc7b2f84b)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:17.811656 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:17.858743 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78fedd985d3b4bebbf0aefecc7b2f84b', 'request_id': 'req-7f45b925-d3cd-4897-8a64-5f868f36c64b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:17.864704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78fedd985d3b4bebbf0aefecc7b2f84b', 'request_id': 'req-7f45b925-d3cd-4897-8a64-5f868f36c64b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:17.872080 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78fedd985d3b4bebbf0aefecc7b2f84b', 'request_id': 'req-7f45b925-d3cd-4897-8a64-5f868f36c64b'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:17.882214 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] The token cache is being invalidate because project 78fedd985d3b4bebbf0aefecc7b2f84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:18:17.882500 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:17.883119 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:17.884679 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1430/2867] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:18:17 2022] DELETE /identity/v3/projects/78fedd985d3b4bebbf0aefecc7b2f84b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:17.889723 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607e3e1-f5de-4794-9fcd-b413a806a237 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:17.899256 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607e3e1-f5de-4794-9fcd-b413a806a237 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:17.956303 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:17.957201 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:17.957485 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:17.957764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] PATH_INFO: `/v3/users/a6f10b01611842d0bda15a1c669b0154` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:17.958933 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f10b01611842d0bda15a1c669b0154)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:17.960898 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:17.970792 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:18.006431 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f10b01611842d0bda15a1c669b0154'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:18.007076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f10b01611842d0bda15a1c669b0154'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:18.014108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f10b01611842d0bda15a1c669b0154'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:18.014728 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f10b01611842d0bda15a1c669b0154'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:18.026419 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-734ce2da-b8e3-4f03-8349-274242a47f1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f10b01611842d0bda15a1c669b0154'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:18.038550 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1438/2868] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:18:17 2022] DELETE /identity/v3/users/a6f10b01611842d0bda15a1c669b0154 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:18.321326 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:18.322754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:18.336661 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45b925-d3cd-4897-8a64-5f868f36c64b admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:18.397286 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:18.398399 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:18.398655 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:18.398935 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] PATH_INFO: `/v3/projects/d9c77ac564434ac8b4756d362436fab5` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:18.400135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9c77ac564434ac8b4756d362436fab5)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:18.402084 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:18.414762 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:18.452077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c77ac564434ac8b4756d362436fab5', 'request_id': 'req-ff7976cc-7c2c-45bb-967a-51f314428482'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:18.458142 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c77ac564434ac8b4756d362436fab5', 'request_id': 'req-ff7976cc-7c2c-45bb-967a-51f314428482'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:18.464657 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c77ac564434ac8b4756d362436fab5', 'request_id': 'req-ff7976cc-7c2c-45bb-967a-51f314428482'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:18.475489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] The token cache is being invalidate because project d9c77ac564434ac8b4756d362436fab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:18:18.475774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:18.476386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-ff7976cc-7c2c-45bb-967a-51f314428482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:18.477987 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1431/2869] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:18:18 2022] DELETE /identity/v3/projects/d9c77ac564434ac8b4756d362436fab5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:18.483710 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005b674-8d91-4ca5-b00d-6fec63cdbb3d None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:18.484237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005b674-8d91-4ca5-b00d-6fec63cdbb3d None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:18.484352 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005b674-8d91-4ca5-b00d-6fec63cdbb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:18.502478 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-4005b674-8d91-4ca5-b00d-6fec63cdbb3d None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:18.503688 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-4005b674-8d91-4ca5-b00d-6fec63cdbb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:18.562875 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1439/2870] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:18.570429 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d9582-4b0d-4db2-832f-5403445fa8be None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:18.570712 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d9582-4b0d-4db2-832f-5403445fa8be None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:18.570982 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d9582-4b0d-4db2-832f-5403445fa8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:18.589246 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-c27d9582-4b0d-4db2-832f-5403445fa8be None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:18.590437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-c27d9582-4b0d-4db2-832f-5403445fa8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:18.598527 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1432/2871] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:18.602693 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005b674-8d91-4ca5-b00d-6fec63cdbb3d None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:18.613054 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005b674-8d91-4ca5-b00d-6fec63cdbb3d None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:18.617671 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f5e37-47b6-4967-b91f-249e0c649ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:18.618326 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f5e37-47b6-4967-b91f-249e0c649ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:18.618614 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f5e37-47b6-4967-b91f-249e0c649ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:18.618894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f5e37-47b6-4967-b91f-249e0c649ce1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:18.619625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f5e37-47b6-4967-b91f-249e0c649ce1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 08 02:18:18.619958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f5e37-47b6-4967-b91f-249e0c649ce1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:18.621594 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f5e37-47b6-4967-b91f-249e0c649ce1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:18.633841 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1440/2872] 10.222.0.20 () {58 vars in 1152 bytes} [Wed Jun 8 02:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 08 02:18:18.638155 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d9582-4b0d-4db2-832f-5403445fa8be None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:18.647372 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d9582-4b0d-4db2-832f-5403445fa8be None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:18.652029 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497b687-9e6e-44fd-b469-517c68bf4d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:18.652704 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497b687-9e6e-44fd-b469-517c68bf4d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:18.652986 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497b687-9e6e-44fd-b469-517c68bf4d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:18.653260 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497b687-9e6e-44fd-b469-517c68bf4d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:18.654205 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497b687-9e6e-44fd-b469-517c68bf4d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:18.656077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497b687-9e6e-44fd-b469-517c68bf4d24 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:18.681810 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1433/2873] 10.222.0.20 () {60 vars in 1152 bytes} [Wed Jun 8 02:18:18 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 08 02:18:18.686571 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f5e37-47b6-4967-b91f-249e0c649ce1 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:18.696714 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f5e37-47b6-4967-b91f-249e0c649ce1 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:18.752285 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651fc2c-81f8-4354-b89f-dd2ff9e65cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:18.752982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651fc2c-81f8-4354-b89f-dd2ff9e65cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:18.753267 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651fc2c-81f8-4354-b89f-dd2ff9e65cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:18.753577 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651fc2c-81f8-4354-b89f-dd2ff9e65cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:18.754274 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651fc2c-81f8-4354-b89f-dd2ff9e65cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:18.756183 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651fc2c-81f8-4354-b89f-dd2ff9e65cc8 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:18.775704 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1441/2874] 10.222.0.20 () {60 vars in 1143 bytes} [Wed Jun 8 02:18:18 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 08 02:18:18.779871 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497b687-9e6e-44fd-b469-517c68bf4d24 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:18.790306 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497b687-9e6e-44fd-b469-517c68bf4d24 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:18.795652 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c63906-c478-42b6-8b9e-7acac3c831c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:18.796310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63906-c478-42b6-8b9e-7acac3c831c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:18.796644 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63906-c478-42b6-8b9e-7acac3c831c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:18.796903 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63906-c478-42b6-8b9e-7acac3c831c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:18.797625 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c63906-c478-42b6-8b9e-7acac3c831c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:18.799302 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c63906-c478-42b6-8b9e-7acac3c831c7 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:18.811283 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1434/2875] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:18:18.815416 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651fc2c-81f8-4354-b89f-dd2ff9e65cc8 admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:18.821828 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:18.826063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651fc2c-81f8-4354-b89f-dd2ff9e65cc8 admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:18.832253 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b014380-68e5-4295-8268-ece041b144f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:18.833108 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b014380-68e5-4295-8268-ece041b144f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:18.833464 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b014380-68e5-4295-8268-ece041b144f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:18.833913 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b014380-68e5-4295-8268-ece041b144f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:18.834662 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b014380-68e5-4295-8268-ece041b144f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:18.836269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b014380-68e5-4295-8268-ece041b144f1 admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:18.846873 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1442/2876] 10.222.0.20 () {58 vars in 1121 bytes} [Wed Jun 8 02:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 08 02:18:18.850808 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c63906-c478-42b6-8b9e-7acac3c831c7 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:18.860763 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c63906-c478-42b6-8b9e-7acac3c831c7 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:18.865412 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d242bb2-ad1f-4cc3-9951-e8710651f657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:18.866287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d242bb2-ad1f-4cc3-9951-e8710651f657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:18.866574 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d242bb2-ad1f-4cc3-9951-e8710651f657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:18.866870 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d242bb2-ad1f-4cc3-9951-e8710651f657 admin admin] PATH_INFO: `/v3/projects/741042da56d346c8993f525a73c443f9/users/eafd60434a7c4a8b83c77227949bc3ff/roles/61490cf1d7334421b84ff67552b79bfc` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:18.881774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d242bb2-ad1f-4cc3-9951-e8710651f657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=741042da56d346c8993f525a73c443f9, user_id=eafd60434a7c4a8b83c77227949bc3ff, role_id=61490cf1d7334421b84ff67552b79bfc)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:18.884450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d242bb2-ad1f-4cc3-9951-e8710651f657 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:18.898056 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1435/2877] 10.222.0.20 () {60 vars in 1482 bytes} [Wed Jun 8 02:18:18 2022] PUT /identity/v3/projects/741042da56d346c8993f525a73c443f9/users/eafd60434a7c4a8b83c77227949bc3ff/roles/61490cf1d7334421b84ff67552b79bfc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 08 02:18:18.909338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761ffb7-fc4b-4e36-8195-f7c7f3f3ed60 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:18.909641 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761ffb7-fc4b-4e36-8195-f7c7f3f3ed60 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:18.909927 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761ffb7-fc4b-4e36-8195-f7c7f3f3ed60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:18.939523 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-d761ffb7-fc4b-4e36-8195-f7c7f3f3ed60 None None] MFA Rules not processed for user `eafd60434a7c4a8b83c77227949bc3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:18.940770 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-d761ffb7-fc4b-4e36-8195-f7c7f3f3ed60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:19.004467 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1443/2878] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:19.015493 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d242bb2-ad1f-4cc3-9951-e8710651f657 admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:19.017298 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-5d242bb2-ad1f-4cc3-9951-e8710651f657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:19.075146 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d242bb2-ad1f-4cc3-9951-e8710651f657 admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:19.079388 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12dc25-d171-4c1e-907f-5a2f21d6e281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '236226ec865b49febc7713df1c0fab98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:19.080038 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12dc25-d171-4c1e-907f-5a2f21d6e281 service cinder] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:19.080324 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12dc25-d171-4c1e-907f-5a2f21d6e281 service cinder] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:19.080632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12dc25-d171-4c1e-907f-5a2f21d6e281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:19.088857 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12dc25-d171-4c1e-907f-5a2f21d6e281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:19.090295 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12dc25-d171-4c1e-907f-5a2f21d6e281 service cinder] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:19.099458 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1436/2879] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:18:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:18:20.446410 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436673-abd8-4537-88a1-73d0cd134d2b None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:20.446890 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436673-abd8-4537-88a1-73d0cd134d2b None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:20.447018 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436673-abd8-4537-88a1-73d0cd134d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:20.464868 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-99436673-abd8-4537-88a1-73d0cd134d2b None None] MFA Rules not processed for user `eafd60434a7c4a8b83c77227949bc3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:20.466076 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-99436673-abd8-4537-88a1-73d0cd134d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:20.472950 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1444/2880] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:29.095468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:30.459580 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=38 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:32.935287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12dc25-d171-4c1e-907f-5a2f21d6e281 service cinder] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:32.936811 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-0a12dc25-d171-4c1e-907f-5a2f21d6e281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:33.034130 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12dc25-d171-4c1e-907f-5a2f21d6e281 service cinder] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:33.039360 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339c14b-91b0-4711-9f69-aa32a5c8e7a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:33.040005 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339c14b-91b0-4711-9f69-aa32a5c8e7a4 service placement] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:33.040287 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339c14b-91b0-4711-9f69-aa32a5c8e7a4 service placement] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:33.040599 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339c14b-91b0-4711-9f69-aa32a5c8e7a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:33.042338 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d339c14b-91b0-4711-9f69-aa32a5c8e7a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:33.054303 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339c14b-91b0-4711-9f69-aa32a5c8e7a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:33.055885 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339c14b-91b0-4711-9f69-aa32a5c8e7a4 service placement] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:33.066484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1437/2881] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:18:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:18:34.155493 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f95119-946d-45ec-9812-0118a25d7c36 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:34.155963 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f95119-946d-45ec-9812-0118a25d7c36 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:34.156083 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f95119-946d-45ec-9812-0118a25d7c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:34.212610 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-67f95119-946d-45ec-9812-0118a25d7c36 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:34.213747 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-67f95119-946d-45ec-9812-0118a25d7c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:34.286958 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1445/2882] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:18:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:34.333507 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339c14b-91b0-4711-9f69-aa32a5c8e7a4 service placement] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:34.334946 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-d339c14b-91b0-4711-9f69-aa32a5c8e7a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:34.425912 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339c14b-91b0-4711-9f69-aa32a5c8e7a4 service placement] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:34.430883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d412863-9ef7-4821-aa1b-e68d1054f8e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:34.431526 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d412863-9ef7-4821-aa1b-e68d1054f8e8 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:34.431807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d412863-9ef7-4821-aa1b-e68d1054f8e8 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:34.432077 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d412863-9ef7-4821-aa1b-e68d1054f8e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:34.439975 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d412863-9ef7-4821-aa1b-e68d1054f8e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:34.441617 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d412863-9ef7-4821-aa1b-e68d1054f8e8 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:34.449492 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1438/2883] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:18:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:18:34.962837 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26a8d4-08ff-4ba6-b803-1b48b6c3adc0 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:34.963132 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26a8d4-08ff-4ba6-b803-1b48b6c3adc0 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:34.963412 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26a8d4-08ff-4ba6-b803-1b48b6c3adc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:34.981611 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-bb26a8d4-08ff-4ba6-b803-1b48b6c3adc0 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:34.982731 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-bb26a8d4-08ff-4ba6-b803-1b48b6c3adc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:34.990474 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1446/2884] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jun 8 02:18:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:35.002320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d412863-9ef7-4821-aa1b-e68d1054f8e8 service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:35.013340 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d412863-9ef7-4821-aa1b-e68d1054f8e8 service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:35.018604 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4874537-1c8d-4efd-abe5-6db0ed164599 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:35.019242 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874537-1c8d-4efd-abe5-6db0ed164599 service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:35.019529 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874537-1c8d-4efd-abe5-6db0ed164599 service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:35.019804 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874537-1c8d-4efd-abe5-6db0ed164599 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:35.027802 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4874537-1c8d-4efd-abe5-6db0ed164599 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:35.029414 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4874537-1c8d-4efd-abe5-6db0ed164599 service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:35.038437 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1439/2885] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:18:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:18:35.226619 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f655d46-fd2d-4ce5-b545-d290dca6d051 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:35.227099 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f655d46-fd2d-4ce5-b545-d290dca6d051 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:35.227237 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f655d46-fd2d-4ce5-b545-d290dca6d051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:35.246246 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-1f655d46-fd2d-4ce5-b545-d290dca6d051 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:35.247377 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-1f655d46-fd2d-4ce5-b545-d290dca6d051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:35.255333 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1447/2886] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Jun 8 02:18:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:35.267445 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4874537-1c8d-4efd-abe5-6db0ed164599 service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:35.277910 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4874537-1c8d-4efd-abe5-6db0ed164599 service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:35.282942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e7d60-1ab6-4ab2-8c34-d3bf73e582ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:35.283579 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e7d60-1ab6-4ab2-8c34-d3bf73e582ad service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:35.283864 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e7d60-1ab6-4ab2-8c34-d3bf73e582ad service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:35.284140 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e7d60-1ab6-4ab2-8c34-d3bf73e582ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:35.292370 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e7d60-1ab6-4ab2-8c34-d3bf73e582ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:35.293970 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e7d60-1ab6-4ab2-8c34-d3bf73e582ad service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:35.302032 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1440/2887] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:18:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:18:35.619708 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c1bb7-5214-466f-899f-17585c600008 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:35.620002 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c1bb7-5214-466f-899f-17585c600008 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:35.620279 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c1bb7-5214-466f-899f-17585c600008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:35.638424 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-361c1bb7-5214-466f-899f-17585c600008 None None] MFA Rules not processed for user `a6aa350eb1504e9c9be9c90870779381`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:35.639720 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-361c1bb7-5214-466f-899f-17585c600008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:35.646858 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1448/2888] 10.222.0.20 () {58 vars in 967 bytes} [Wed Jun 8 02:18:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:35.658528 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e7d60-1ab6-4ab2-8c34-d3bf73e582ad service swift] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:35.669509 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e7d60-1ab6-4ab2-8c34-d3bf73e582ad service swift] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:35.675036 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1740c0-aafc-414d-8b93-5bb00c561c7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc299258fe814636880cf595fc2904b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:35.675673 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1740c0-aafc-414d-8b93-5bb00c561c7b service swift] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:35.675955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1740c0-aafc-414d-8b93-5bb00c561c7b service swift] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:35.676227 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1740c0-aafc-414d-8b93-5bb00c561c7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:35.684336 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1740c0-aafc-414d-8b93-5bb00c561c7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:35.685968 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1740c0-aafc-414d-8b93-5bb00c561c7b service swift] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:35.690632 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=100 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:35.694825 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1441/2889] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Jun 8 02:18:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 08 02:18:37.914625 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c1bb7-5214-466f-899f-17585c600008 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:37.978124 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c1bb7-5214-466f-899f-17585c600008 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:37.982606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:37.983961 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:37.984243 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:37.984566 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] PATH_INFO: `/v3/users/eafd60434a7c4a8b83c77227949bc3ff` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:37.985749 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] RBAC: Authorizing `identity:delete_user(user_id=eafd60434a7c4a8b83c77227949bc3ff)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:37.987685 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:38.035434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:38.042560 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eafd60434a7c4a8b83c77227949bc3ff'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:38.043233 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eafd60434a7c4a8b83c77227949bc3ff'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:38.050288 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eafd60434a7c4a8b83c77227949bc3ff'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:38.050923 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eafd60434a7c4a8b83c77227949bc3ff'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:38.063282 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eafd60434a7c4a8b83c77227949bc3ff'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:38.075985 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1449/2890] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:18:37 2022] DELETE /identity/v3/users/eafd60434a7c4a8b83c77227949bc3ff => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:38.081434 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d3fb5-dafc-4f02-83cf-fe30b0f3b0fa None None] REQUEST_METHOD: `POST` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:38.082105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d3fb5-dafc-4f02-83cf-fe30b0f3b0fa None None] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:38.082105 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d3fb5-dafc-4f02-83cf-fe30b0f3b0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:38.099637 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.auth.core [None req-1c7d3fb5-dafc-4f02-83cf-fe30b0f3b0fa None None] MFA Rules not processed for user `ded448ce0e8547839095b6fee4b26e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:18:38.100911 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1c7d3fb5-dafc-4f02-83cf-fe30b0f3b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:38.159920 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1442/2891] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:18:38.169612 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:38.171012 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:38.227604 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8cf25-08ea-46b3-8e4f-47edde81bead admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:38.231241 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b07ca-919e-4e24-946d-68f169a017e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:38.231878 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b07ca-919e-4e24-946d-68f169a017e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:38.232162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b07ca-919e-4e24-946d-68f169a017e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:38.232466 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b07ca-919e-4e24-946d-68f169a017e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:38.241521 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b07ca-919e-4e24-946d-68f169a017e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:38.242951 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b07ca-919e-4e24-946d-68f169a017e4 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:38.252741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1450/2892] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:18:38.438377 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d3fb5-dafc-4f02-83cf-fe30b0f3b0fa None None] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:18:38.439820 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-1c7d3fb5-dafc-4f02-83cf-fe30b0f3b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:18:38.453135 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d3fb5-dafc-4f02-83cf-fe30b0f3b0fa None None] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:18:38.458473 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:18:38.459355 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:18:38.459649 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:18:38.459990 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] PATH_INFO: `/v3/projects/741042da56d346c8993f525a73c443f9` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:18:38.461200 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=741042da56d346c8993f525a73c443f9)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:18:38.463213 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:18:38.516156 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741042da56d346c8993f525a73c443f9', 'request_id': 'req-7aff5021-e35f-40d6-a280-f2191823c6ea'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:38.522194 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741042da56d346c8993f525a73c443f9', 'request_id': 'req-7aff5021-e35f-40d6-a280-f2191823c6ea'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:38.528868 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741042da56d346c8993f525a73c443f9', 'request_id': 'req-7aff5021-e35f-40d6-a280-f2191823c6ea'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:38.539858 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] The token cache is being invalidate because project 741042da56d346c8993f525a73c443f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:18:38.540141 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:38.542895 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:18:38.544477 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1443/2893] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:18:38 2022] DELETE /identity/v3/projects/741042da56d346c8993f525a73c443f9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:18:48.248014 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:18:48.536821 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:19:09.064699 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab6bc3-8b19-4bce-97d0-ce830b15cbd1 None None] REQUEST_METHOD: `POST` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:19:09.065269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab6bc3-8b19-4bce-97d0-ce830b15cbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:19:09.065269 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab6bc3-8b19-4bce-97d0-ce830b15cbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:19:09.082585 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.auth.core [None req-a3ab6bc3-8b19-4bce-97d0-ce830b15cbd1 None None] MFA Rules not processed for user `89e6c19fcc9f490c890fc4d6350273a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 08 02:19:09.083782 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a3ab6bc3-8b19-4bce-97d0-ce830b15cbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:19:09.137260 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1451/2894] 10.222.0.20 () {58 vars in 957 bytes} [Wed Jun 8 02:19:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 08 02:19:19.130791 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=25 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:19:26.001160 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:19:26.002724 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:19:26.101997 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff5021-e35f-40d6-a280-f2191823c6ea admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:19:26.105807 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dd06c-eded-46da-9f04-5218c31d5dcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ef930dc7b7a4fb29f7b32e464e9eb3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:19:26.106468 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dd06c-eded-46da-9f04-5218c31d5dcb service nova] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:19:26.106754 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dd06c-eded-46da-9f04-5218c31d5dcb service nova] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:19:26.107031 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dd06c-eded-46da-9f04-5218c31d5dcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:19:26.108883 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-389dd06c-eded-46da-9f04-5218c31d5dcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:19:26.121505 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dd06c-eded-46da-9f04-5218c31d5dcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:19:26.123111 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dd06c-eded-46da-9f04-5218c31d5dcb service nova] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:19:26.134152 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1444/2895] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:19:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:19:36.129293 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=43 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:21:04.807640 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab6bc3-8b19-4bce-97d0-ce830b15cbd1 None None] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:21:04.809085 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-a3ab6bc3-8b19-4bce-97d0-ce830b15cbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:21:04.874228 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab6bc3-8b19-4bce-97d0-ce830b15cbd1 None None] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:21:04.879764 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839d2c-3735-4086-a403-84dcb9a1ecb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:21:04.880441 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839d2c-3735-4086-a403-84dcb9a1ecb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:21:04.880727 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839d2c-3735-4086-a403-84dcb9a1ecb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:21:04.881004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839d2c-3735-4086-a403-84dcb9a1ecb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:21:04.883010 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-49839d2c-3735-4086-a403-84dcb9a1ecb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:21:04.896162 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839d2c-3735-4086-a403-84dcb9a1ecb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:21:04.897666 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839d2c-3735-4086-a403-84dcb9a1ecb4 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:21:04.907002 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1452/2896] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:21:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:21:05.911809 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dd06c-eded-46da-9f04-5218c31d5dcb service nova] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:21:05.913310 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-389dd06c-eded-46da-9f04-5218c31d5dcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:21:05.926291 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dd06c-eded-46da-9f04-5218c31d5dcb service nova] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:21:05.933322 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428263d4-98e6-488b-90c8-23d0432918ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:21:05.934484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428263d4-98e6-488b-90c8-23d0432918ed service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:21:05.934484 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428263d4-98e6-488b-90c8-23d0432918ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:21:05.934702 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428263d4-98e6-488b-90c8-23d0432918ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:21:05.937615 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-428263d4-98e6-488b-90c8-23d0432918ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:21:05.953149 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428263d4-98e6-488b-90c8-23d0432918ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:21:05.954838 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428263d4-98e6-488b-90c8-23d0432918ed service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:21:05.967847 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1445/2897] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:21:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:21:07.157271 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839d2c-3735-4086-a403-84dcb9a1ecb4 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:21:07.166536 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839d2c-3735-4086-a403-84dcb9a1ecb4 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:21:07.170226 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c88ac-7980-41f5-8dda-333307a94399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:21:07.170896 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c88ac-7980-41f5-8dda-333307a94399 service neutron] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:21:07.171184 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c88ac-7980-41f5-8dda-333307a94399 service neutron] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:21:07.171483 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c88ac-7980-41f5-8dda-333307a94399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:21:07.173398 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-db5c88ac-7980-41f5-8dda-333307a94399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:21:07.188355 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c88ac-7980-41f5-8dda-333307a94399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:21:07.189832 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c88ac-7980-41f5-8dda-333307a94399 service neutron] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:21:07.201579 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1453/2898] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:21:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:21:15.962320 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=6 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:21:17.196642 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=32 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:21:18.439875 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428263d4-98e6-488b-90c8-23d0432918ed service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:21:18.448696 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428263d4-98e6-488b-90c8-23d0432918ed service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:21:18.452300 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca13c7-d2bd-4086-b022-3a4ac3022702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd26f4426ba8c43399c4a7ef2f1701f57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:21:18.452992 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca13c7-d2bd-4086-b022-3a4ac3022702 service neutron] REQUEST_METHOD: `GET` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:21:18.453280 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca13c7-d2bd-4086-b022-3a4ac3022702 service neutron] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:21:18.453553 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca13c7-d2bd-4086-b022-3a4ac3022702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:21:18.455346 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-b9ca13c7-d2bd-4086-b022-3a4ac3022702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:21:18.521062 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca13c7-d2bd-4086-b022-3a4ac3022702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:21:18.522508 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca13c7-d2bd-4086-b022-3a4ac3022702 service neutron] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:21:18.532163 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1446/2899] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:21:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:21:24.446091 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c88ac-7980-41f5-8dda-333307a94399 service neutron] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:21:24.447552 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-db5c88ac-7980-41f5-8dda-333307a94399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:21:24.461713 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c88ac-7980-41f5-8dda-333307a94399 service neutron] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:21:24.466606 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:21:24.467583 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:21:24.467982 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:21:24.468741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] PATH_INFO: `/v3/users/89e6c19fcc9f490c890fc4d6350273a0` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:21:24.470045 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=89e6c19fcc9f490c890fc4d6350273a0)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:21:24.472389 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:21:24.530222 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89e6c19fcc9f490c890fc4d6350273a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:24.531123 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89e6c19fcc9f490c890fc4d6350273a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:24.540114 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89e6c19fcc9f490c890fc4d6350273a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:24.540869 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89e6c19fcc9f490c890fc4d6350273a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:24.555004 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89e6c19fcc9f490c890fc4d6350273a0'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:24.571340 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1454/2900] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:21:24 2022] DELETE /identity/v3/users/89e6c19fcc9f490c890fc4d6350273a0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:21:24.746369 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca13c7-d2bd-4086-b022-3a4ac3022702 service neutron] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:21:24.758270 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca13c7-d2bd-4086-b022-3a4ac3022702 service neutron] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:21:24.817861 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:21:24.818640 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:21:24.818926 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:21:24.819197 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] PATH_INFO: `/v3/projects/079027b24aba4a179f99de9f6fe944b8` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:21:24.820381 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] RBAC: Authorizing `identity:delete_project(project_id=079027b24aba4a179f99de9f6fe944b8)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:21:24.822325 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:21:24.869942 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '079027b24aba4a179f99de9f6fe944b8', 'request_id': 'req-74c87efa-db92-4fd9-a580-ffdeca50425c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:24.876167 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '079027b24aba4a179f99de9f6fe944b8', 'request_id': 'req-74c87efa-db92-4fd9-a580-ffdeca50425c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:24.883691 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '079027b24aba4a179f99de9f6fe944b8', 'request_id': 'req-74c87efa-db92-4fd9-a580-ffdeca50425c'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:24.894489 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] The token cache is being invalidate because project 079027b24aba4a179f99de9f6fe944b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:21:24.894774 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:24.895386 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:24.896955 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1447/2901] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:21:24 2022] DELETE /identity/v3/projects/079027b24aba4a179f99de9f6fe944b8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:21:24.900978 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:21:24.902380 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:21:24.916434 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ea5f7-72b8-438d-8857-6bf6869eb3bb admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:21:24.972613 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:21:24.973496 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:21:24.973783 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:21:24.974063 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] PATH_INFO: `/v3/users/08b187e774f745418614c327e14d8c31` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:21:24.975258 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] RBAC: Authorizing `identity:delete_user(user_id=08b187e774f745418614c327e14d8c31)` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:21:24.977218 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:21:25.023293 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08b187e774f745418614c327e14d8c31'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:25.023940 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08b187e774f745418614c327e14d8c31'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:25.030760 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08b187e774f745418614c327e14d8c31'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:25.031384 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08b187e774f745418614c327e14d8c31'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:25.043906 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.notifications [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08b187e774f745418614c327e14d8c31'} {{(pid=80426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:25.045741 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:21:25.060338 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1455/2902] 10.222.0.20 () {58 vars in 1223 bytes} [Wed Jun 8 02:21:24 2022] DELETE /identity/v3/users/08b187e774f745418614c327e14d8c31 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:21:25.258681 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] Authenticating user token {{(pid=80425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:21:25.260090 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.fernet_utils [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:21:25.274316 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c87efa-db92-4fd9-a580-ffdeca50425c admin admin] Validating token access rules against request {{(pid=80425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:21:25.300571 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:21:25.341450 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ded448ce0e8547839095b6fee4b26e38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd34d18aa450b41d2ac1001da2d9a5476', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:21:25.342321 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:21:25.342604 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:21:25.342878 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] PATH_INFO: `/v3/projects/d56f3fbee70c4c9d98af73d13cbd9a0e` {{(pid=80425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:21:25.344053 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56f3fbee70c4c9d98af73d13cbd9a0e)` {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:21:25.345991 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] RBAC: Authorization granted {{(pid=80425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:21:25.393443 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56f3fbee70c4c9d98af73d13cbd9a0e', 'request_id': 'req-daf9289d-4d6b-4af5-b309-47d57d609af4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:25.399378 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56f3fbee70c4c9d98af73d13cbd9a0e', 'request_id': 'req-daf9289d-4d6b-4af5-b309-47d57d609af4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:25.406837 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56f3fbee70c4c9d98af73d13cbd9a0e', 'request_id': 'req-daf9289d-4d6b-4af5-b309-47d57d609af4'} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:25.418362 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] The token cache is being invalidate because project d56f3fbee70c4c9d98af73d13cbd9a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 08 02:21:25.418659 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:25.419279 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG keystone.notifications [None req-daf9289d-4d6b-4af5-b309-47d57d609af4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 08 02:21:25.420894 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: [pid: 80425|app: 0|req: 1448/2903] 10.222.0.20 () {58 vars in 1232 bytes} [Wed Jun 8 02:21:25 2022] DELETE /identity/v3/projects/d56f3fbee70c4c9d98af73d13cbd9a0e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 08 02:21:35.060823 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:21:35.415301 dv-cinder-845069-1-iscsi devstack@keystone.service[80425]: DEBUG dbcounter [-] [80425] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 08 02:23:42.621035 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] Authenticating user token {{(pid=80426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 08 02:23:42.622430 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:23:42.697500 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223a773-dbb5-47c9-beb1-62c0b938360b admin admin] Validating token access rules against request {{(pid=80426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 08 02:23:42.755897 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc7a5b-141f-46db-9bda-ea4bd0a791e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eb69af145154c6d85bd4b845979e6c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fed22cd0ec848c88e021eff03a083f0', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 08 02:23:42.756565 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc7a5b-141f-46db-9bda-ea4bd0a791e7 service placement] REQUEST_METHOD: `GET` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 08 02:23:42.756854 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc7a5b-141f-46db-9bda-ea4bd0a791e7 service placement] SCRIPT_NAME: `/identity` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 08 02:23:42.757131 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc7a5b-141f-46db-9bda-ea4bd0a791e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 08 02:23:42.758894 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.fernet_utils [None req-17cc7a5b-141f-46db-9bda-ea4bd0a791e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 08 02:23:42.770129 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc7a5b-141f-46db-9bda-ea4bd0a791e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 08 02:23:42.771714 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc7a5b-141f-46db-9bda-ea4bd0a791e7 service placement] RBAC: Authorization granted {{(pid=80426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 08 02:23:42.781597 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: [pid: 80426|app: 0|req: 1456/2904] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Jun 8 02:23:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 08 02:23:52.777314 dv-cinder-845069-1-iscsi devstack@keystone.service[80426]: DEBUG dbcounter [-] [80426] Writing DB stats keystone:SELECT=58 {{(pid=80426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}